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(U)  Analysis  of  Crew/ Cockpit  Models  for  Advanced 
Aircraft,  by  Charles  P.  Greening,  Autoneclcs  Division, 
Rockwell  International.  China  Lake,  Calif.,  Naval 
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(uJflThe  purpose  of  this  study  was  to  examine 
five  active  computer  models  of  the  aircrew/ cockpit 
system,  and  to  determine  their  relevance  to  current 
and  future  attack  aircraft.  The  models  were  compared 
in  terms  of  their  general  structure,  input  requirements, 
output  options,  and  their  sensitivity  to  a wide  variety 
of  equipment,  mission,  and  operator  characteristics.  ^ 
(U)  ^The  models  reviewed  'are  similar  in  several 
Important  respects:  They  all  require  a detailed 
mission  scenario  and  task  analysis  to  steer  the  simu- 
lation; all  require  data  on  performance  time  and 
accuracy  as  inputs;  and  all  generate  outputs  related 
to  operator  task  load.  The  models  differ  widely  in 
their  sensitivity  to  significant  variables.  Recom- 
mendations for  future  work  are  presented. 
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INTRODUCTION 

Advanced  aircraft  systems  are  Increasingly  placing  the  crew  member 
In  the  role  of  a system  monitor  or  supervisor,  rather  than  a simple 
detector,  power  amplifier,  and  actuator.  This  kind  of  assertion  has 
become  a platitude  In  recent  years.  But  as  recently  as  March  of  1976, 
participants  In  an  International  symposium  on  monitoring  and  super- 
visory behavior  concluded  that 

"Models  of  human  monitoring  and  supervisory 
control  behavior,  human  Information  processing 
and  decision-making  exist,  but  are  often 
primitive  or  not  well  matched  to  applications. 

The  papers  presented  at  that  symposium  present  a broad  range  of 
suggested  approaches  to  the  problem  of  modeling  these  characteristi- 
cally human  processes.  The  stunning  variety  of  the  approaches  gives 
some  insight  into  the  degree  of  uncertainty  among  specialists  as  to  the 
"best"  model  of  the  operator-as-monitor  or  -supervisor. 

There  are  basically  two  ways  to  approach  the  problem  of  modeling 
the  crew/cockpit  system  in  the  absence  of  widely  accepted  models  of  the 
supervisory  or  monitoring  portions  of  the  crew  task:  (1)  model  those 
portions  we  can,  and  hope  that  the  omissions  and/or  approximations 
don't  matter  too  much  or  (2)  wait  until  we  can  settle  on  an  adequate 
model  of  the  "higher"  functions.  The  choice  between  these  alternatives 
is  often  forced  by  the  urgency  of  the  need  to  have  some  kind  of  model 
operating  - especially  among  designers/producers  of  new  aircraft 
systems.  Hence,  the  "finished,"  operating  models  tend  to  represent  the 
first  approach. 

The  situation  is.  In  a sense,  parallel  with  the  modeling  of  visual 
target  acquisition.  Most  of  those  models  emphasize  such  functions  as 
visual  acuity  and  the  detection  of  objects  in  a featureless  or  clutter- 
ed background,  while  minimizing  or  omitting  such  activities  and  vari- 
ables as  search  strategy,  training,  briefing  and  experience.? 


T.B.  Sheridan  and  G.  Johannsen.  "Workshop  Reports.  Introduction  and 
Summary"  in  Monitoring  Behavior  and  Supervisory  Control,  ed.  by  T.3. 
Sheridan  and  G.  Johannsen.  NATO  Conference  Series,  III  Human  Factors, 
Volume  L.  New  York,  Plenum  Press,  1976.  pp  473-77. 

"C.P.  Greening.  "Mathematical  Modeling  of  Air-co-Ground  Target  Acqui- 
sition," Human  Factors,  Vol.  13,  No.  2 (April  1976),  ?p  lil-l-td. 
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The  reader  should  bear  la  mind,  In  Che  descrip dona  and  compari- 
sons which  follow,  thee  Che  anchors  of  Che  coopleced,  documented  models 
have  perforce  modeled  Chose  characteristics  which  are  understood,  and 
simplified  or  omlcted  the  remainder. 


SELECTION  OF  MODELS 

The  current  Statement  of  Work  for  this  study  listed  four  modeling 
efforts  to  be  Included3:  Naval  Air  Development  Center;  McDonnell 
Douglas,  St.  Louis;  LTV  Aerospace;  and  Boeing,  Seattle.  All  four  of 
chose  listed  have  provided  documentation  on  their  respective  models. 

The  descriptive  material  In  later  sections  of  this  report  Is  based  upon 
those  documents. 

In  addition  to  the  modeling  programs  listed  In  the  contract,  two 
ocher  major  programs  appear  frequently  In  the  literature:  the  SAINT 
effort  at  the  Aero-Medical  Research  Lab,  WPAFB;  and  a long-term  model- 
ing effort  at  Applied  Psychological  Services,  Inc.,  under  various 
sponsorships.  These  two  efforts  were  also  examined  and.  In  the  case 
of  the  Applied  Psychological  Services  work,  included  as  a part  of  the 
review  and  comparison. 

Historically,  the  work  of  Siegel,  Wolf,  and  others  at  Applied 
Psychological  Services  has  influenced  the  ocher  efforts  to  a consider- 
able extent.  The  1961  description  of  what  has  come  to  be  called  the  . 

"Siegel-Wolf  model"4  predates  essentially  all  the  ocher  work  by  several 
years. 

The  SAINT  work  is  not  reviewed  because  SAINT  is  not  a man/cockpit 
model,  as  such.  In  Che  words  of  the  custodian  of  SAINTS 

"Our  philosophy  has  been  to  develop  a fairly 
general  modeling  method  rather  than  promote 

a single  man-model we  believe  it  is  useful 

to  provide  a method  for  building  these  models 
without -dictating  which  model  to  use." 

I 

^Contract  N00123-74-C-0236,  Supplement  No.  6,  August  1,  1976. 

4A. I.  Siegel  and  J.J.  Wolf,  "A  Technique  for  Evaluating  Man/Machine 
System  Designs,"  Human  Factors.  Vol  3,  No.  1 (March  1961)  pp  18-2S. 

e • 

G.P.  Chubb  (Aarospace  Medical  Research  Lab).  Letter  to  Ronald 
Erickson  (NWC),  subject  "SAINT  Documentation,"  24  January  1977. 
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la  the  selection  of  models  for  study,  several  general  criteria 
were  applied,  though  not  rigidly.  The  criteria  ware  either  implied  in 
the  work  statement,  or  developed  during  discussions  with  the  contract 
monitor.  They  include: 

1.  Applicability  to  integrated,  interactive  displays  and 
controls. 

2.  Applicability  to  discrete  as  opposed  to  continuous  tasks. 

(The  continuous  control  modeling  efforts  have  given  rise 
to  e voluminous,  specialized  literature  which  is  not 
examined  here.) 

3.  Reasonable  "complexity."  (i.e..  The  model  must  address 
task  sequences,  multiple  displays  and  controls,  mission 
conditions,  etc. , not  just  single  tasks  or  lumped  task 
load. ) 

4.  Reasonable  "completeness.”  (i.e..  The  model  should  exist 
in  near-usable  form  - not  just  a flow  chart  and  an  idea.) 

5.  Availability  of  documentation. 

I 

DESCRIPTIONS  OF  MODELS 

I 

, In  the  following  pages,  brief,  largely  qualitative  descriptions  of 

five  models  will  be  found.  The  models  will  be  seen  to  have  several 
characteristics  in  common: 

1.  A scenario  and  a related  task  analysis  are  assumed  to 
exist,  providing  the  framework  for  the  simulation. 

This  framework  includes  the  identity  of  the  revised 
tasks,  their  sequence  and  priority,  time  constraints, 
and  crew  assignments. 

2.  Data  on  the  average  or  representative  time  required 

to  perform  each  task  or  subtask,  under  certain  standard 
condition?,  are  required  as  model  inputs. 

3.  Operator  workload  (either  overall  or  part  by  part)  as 
a function  of  time  is  a major  output. 

Although  the  gross  outline  of  each  model  conforms  to  the  pattern 
Implied  by  these  three  characteristics,  the  models  differ  substantially 
In  terms  of  the  way  in  which  system  and  mission  changes  are  reflected, 
and  in  the  perturbing  factors  which  are  considered.  A detailed  com- 
parison of  the  five  models  is  found  in  the  section  following  the  model 
descriptions. 
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The  following  descriptions  arc  presented  In  a consistent  format 
which  Includes  (1)  a general  description,  (2)  a listing  and  description 
of  necessary  inputs,  (3)  a listing  and  description  of  outputs,  and  (4) 
where  appropriate,  a tabular  presentation  of  a typical  cockpit  evalua- 
tion operation. 

THE  SIEGEL-WOLF  MODELS 

Arthur  Siegel,  J.  J.  Wolf,  and  several  associates  at  Applied 
Psychological  Services  have  developed  a group  of  models,  some  of  which 
are  directly  relevant  to  attack  aircraft  operations.  Their  work  has 
had  a strong  Influence  on  most  of  the  other  modeling  efforts  described 
in  this  report. 

The  most  complete  and  available  description  of  the  Siegel-Wolf 
work  is  to  be  found  in  a book  published  in  1969. 6 This  work  describes 
the  "unitary  or  dual -operator  simulation  model"  (which  is  directly 
relevant  to  attack  aircraft)  and  a large-system  model  which  is  most 
relevant  to  shipboard  systems.  The  description  presented  here  is  drawn 
largely  from  the  book,  with  some  additions  from  later  reports  and  per- 
sonal communication  with  the  senior  author.  An  earlier  model,  directed 
specifically  coward  discrete  casks  in  a space  vehicle, ^ is  not  describ- 
ed in  the  book,  but  is  of  sufficient  relevance  to  be  described  in  a 
later  section  of  this  report,  also. 

The  basic  purpose  of  the  "Unitary  or  Dual-Operator  Simulation 
Model"  (U/DOSM)  is  to  provide  information  concerning  the  feasibility 
of  completing  anticipated  missions  within  prescribed  time  limits,  and 
some  idea  of  the  likelihood  of  failure  and  the  degree  of  stress  on  the 
operacor(s).  The  concept  of  "stress  " has  a central  place  in  the  oper- 
ation of  the  model.  Stress  can  arise  from  "falling  behind"  the  requir- 
ed task/ time  sequence,  from  errors  requiring  repetition  of  casks,  from 
equipment  delays  or  from  problems  with  the  ocher  crew  member  in  a dual 
operator  situation. 

As  with  most  man/machine  models,  a task  analysis  must  be  performed 
before  simulation  can  begin.  The  analysis  provides  a list  of  the  input 
data  required  for  each  cask,  and  an  identification  of  necessarily  se- 
quential casks.  Average  task  performance  times  and  standard  deviation 
of  times  and  average  probability  of  success  must  also  be  estimated  for 
each  cask/operacor  combination. 


3 Arthur  I.  Siegel  and  J.  Jay  Wolf.  Man-Machine  Simulation  Models. 
Mew  York,  Wiley-Interscience , 1969. 


' Applied  Psychological  Services.  "A  Discontinuous  Analytic  Model  for 
Simulating  Apollo  Vehicle  Operator  Actions  and  Information  Exchange" 
by  A.  I.  Siegel,  J.J.  Wolf  and  R.  Oilman.  Wavne,  Pa.,  September  1962. 
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aeration  of  U/DOSM 


The  general  simulation  sequence  is  shown  in  Fig.  1,  adapted  from 
Man-Machine  Simulation  Models.  The  simulation  makes  use  of  a hierarchy 
of  "tasks"  and  "subtasks."  The  "cask"  represents  a substantial  segment 
of  a mission,  such  as  making  a carrier  landing.  "Subcasks"  are  more 
unitary  acts,  such  as  setting  a rotary  control,  or  check-reading  a set 
of  instruments.  A task  may  include  several  scores  of  sub tasks. 

The  actual  subtask  simulation  begins  after  point  "c"  and  ends  at 
point  "e"  in  the  flow  chart.  For  the  case  of  a single  operator,  the 
steps  In  subcask  simulation  are  (1)  determining  the  degree  of  urgency  - 
a function  of  remaining  time  available  and  Che  sum  of  the  remaining 
subcask  execution  times;  (2)  calculating  time-stress  level  - from  the 
urgency  condition,  Che  nature  of  the  cask,  and  the  characteristics  of 
the  operator;  (3)  if  Che  subcask  is  a cyclic  one  (e.g.,  watching  a PPI 
radar  strobe),  a delay  until  the  next  available  cycle  time  is  inserted; 
(4)  sub cask  execution  time  is  drawn  from  the  input  distribution  and 
modified  in  accordance  with  stress  level  submodels;  (3)  subcask  success/ 
failure  is  drawn  from  Che  input  success /failure  data,  modified  by  a 
stress  function;  (6)  computing  (from  subcask  performance)  a level  of 
goal  aspiration  which,  in  turn,  affects  the  input  conditions  for  the 
next  subcask;  and  (7)  recording  all  subcask  results,  if  desired.  When 
all  subcasks  in  a given  task  have  been  run,  the  overall  task  data  are 
recorded  and  another  iteration  begun,  until  the  desired  number  of  itera- 
tions is  completed. 


The  steps  in  the  process  are  shown  in  another  formac  in  Table  1 to 
provide  for  direct  comparison  with  the  other  models  described  later. 
Some  of  the  factors  which  enter  only  into  relationships  between  crew 
members  are  deleted. 
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Inputs  Required  by  U/DOSM 


Input  data  required  for  each  subcask  for  each  operator  include: 
subtask  type  (joint,  equipment,  decision,  cyclic,  or  regular);  essen- 
tiality (a  value  between  0 and  1);  precedence  (a  point  in  time  before 
which  the  subcask  cannot  occur,  or  a precedent  cask  by  the  other  opera- 
tor in  a 2-man  crew);  sequence  (the  identity  of  the  next  subtask, 
depending  upon  success  or  failure  of  present  subcask) ; subtask  time 
statistics;  subtask  success  probability  (unperturbed  by  stress  and 
Individual  differences) ; time  required  to  complete  remaining  subcasks 
(essential  and  non-essential);  and  goal  aspiration  level  (a  modifier  to 
performance,  based  in  part  on  preceding  subtask  performance). 


Additional  requirements,  before  simulation  can  begin,  include  four 
parameters  (for  each  operator)  and  several  initial  conditions.  The 
parameters  are  (1)  total  mission  time  limit,  (2)  operator  stress 
threshold,  (3)  operator  individuality  factor  (primarily  a "speed" 
factor),  and  (4)  cycle  time  for  cyclic  tasks. 


I 


1JKE  1.  Siegel-Wolf  model  flow  chart  (adapted  from 
-Machine  Simulation  Models). 


tabi e l.  Breakdown  of  Cockpit  Evaluation  Process  (Siegel-Wolf  Model). 


Step  in  Process 

How  Handle 

External  Co  Model 

Internal 

I.  Define  tasks: 

1.1  Mission  description 

Detailed  list  of  tasks 
& subtasks,  time-based. 

Subtask  ID,  time, 
sequence,  etc., 
entered  inCo 
memory. 

1.2  Cockpit  equipment 
description 

Not  called  for  expli- 
citly. 

Not  represented. 

2.1  Time  required/ cask 

Reference  to  experi- 
mental data  for 
mean , <J . 

Stored  as  trun- 
cated normal 
distributions, 
by  subcask. 

2.2  Accuracy  and/or 

Reference  to  experi- 

Stored  as  P^; 

reliability  of 

mental  data  for 

pass-fail  drawn 

performance 

?subtask ' 

at  random  for  a 
particular  run, 
averaging  out 
to  Pt. 

2.3  Sequence/priority 

Assigned  "essencial- 
ity:"by  subcask # 

Subcasks  can  be 
skipped  as  func- 
tion of  time, 
essentiality . 

3.  Modify  performance 
per  conditions: 

3.1  Reach  distance  & 

D/C  placement 

Reflected  in  2. 1,2. 2, 
if  at  all. 

Not  represented. 

3.2  Effect  of  G-load, 
buffet,  environ- 

Reflected  in  2. 1,2. 2, 
if  at  all . 

Not  represented. 

ment- 

3.3  Effect  of  time 

Reflected  in  time 

Stress  computed 

stress 

allowed  for  total 
cask,  and  in  opera- 
tor "stress  thres- 
hold. " 

for  each  itera- 
tion of  each 
subcask,  basea 
on  tine  remain- 
ing, stress 
threshold,  cask 
type. 
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TABLE  1.  (Continued.) 


Step  in  Process 

How  Handled 

External  to  Model 

Internal 

3.  Modify  performance 
(cont'd) : 

3.4  Effect  of  indiv- 
idual differences 

Assign  values  of: 
-Stress  threshold 
-Individual  "speed" 
-Initial  "Goal  As- 
piration"* 

Compute  current 
stress  level 

(see  3.3),  goal 
aspiration  (from 
past  performance 
stress  level) . 
These  values 
affect  perform- 
ance. 

4.  Determine  Operator 

Task  Load: 

4.1  Time  stress  by 
subtask 

Affected  by  several 
inputs  (see  1.1,  2.1, 
2.3,  3.4). 

Computed  for  each 
subtask  & iter- 
ation; also 
average,  peak, 
final  levels. 

4.2  Operator  Workload 

Same  as  4.1. 

Not  computed  ex- 
plicitly; can 
deduce  from  data 
on  time  overruns 
& waiting  time. 

4.3  Workload  by  mod- 
ality (e.g. , 
hand,  foot) 

No  relevant  inputs. 

No  relevant  out- 
put. 

5.  Determine  Svstem  Per- 
formance: 

5.1  Probability  of 
success 

Affected  by  2.2,  2.3, 
3.3,  3.4  data  inputs. 

1 

Computes  success 
& failure  by 
subtask  by  iter- 
ation; also  sum- 
mary data,  by 
run. 

5.2  Diagnostic 
Information 

Same  as  5.1. 

Failures  can  be 
traced  to  sub- 
tasks 4 stress 

causes. 

;o 
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Initial  conditions  required  included  tha  numbar  of  itaratlons  to 
ba  run,  tha  initial  valua  from  which  subsaquant  (psaudo)  random  numbars 
will  ba  ganaratad,  and  a few  other  identification  and  scaling  values. 

Outputs  from  P/DOSM 

Three  output  options  are  available,  in  addition  to  tha  primary 
Final  Susaary  Tabulation.  Table  2 shows  tha  kinds  of  output  provided 
under  each  option. 

MCDONNELL  DOUGLAS  CORPORATION  PILOT  SIMULATION  MODEL  (PSM) 

The  McDonnell  Douglas  PSM  was  developed  as  one  part  of  a technique 
for  evaluating  alternative  man/machine  designs  in  terms  of  system  per- 
formance, operator  workload,  and  task  distribution  for  hypothetical  air- 
craft systems.  As  described  in  the  abstract  of  an  early  McDonnell 
report:® 

. "It  is  a stochastic,  digital  model  with  variable  and 

parallel  flow  logic  that  allows  simulation  of  simul- 
taneous tasks The  model  has  the  following  advantages 


. Simulates  the  mission  logic 
. Supplies  visual,  right  hand,  left  hand,  feet, 
communication,  and  Information  processing 

task  loading 

. Considers  simultaneous  tasks 
. Provides  task  distributions." 

In  a more  recent  report,  the  PSM  is  described  as  follows 

"The  model  functions  basically  as  an  information  store 
that  is  continually  supplied  with  current  system 

Information subsequently  updated ...  and  provides 

probability  statements  concerning  pilot  activity  as 
an  output." 

The  PSM  is  so  named  because  it  was  originally  developed  to  simu- 
late a pilot's  workload  in  a tactical  fighter  aircraft.  In  the  7 years 
since  its  genesis,  the  model  has  been  refined  and  expanded.  It  is  now 
capable  of  simulating  the  workloads  of  multiple  crews  to  a maximum  of 
100  individually  identified  operators. 

McDonnell  Aircraft  Co.  Digital  Simulation  Model  for  Figncer  Pilot 
Workload , by  C.F.  Asiala,  Jr.  St.  Louis,  MO,  MAC,  September  19o9. 

I.MDC  Report  AO058,  publication  UNCLASSIFIED.) 

9 

Aerospace  Medical  Research  Laboratory.  Advanced  Fighter  Concepts 
Incorporating  High  Acceleration  Cocnoits,  Vol.  d - Pilot  Performance 
Anaayses , by  J.M.  Sinnett  and  C.F.  Asiala.  Wright  ?acter3cn  Air 
Force  dase,  Ohio,  AMRL,  July  1973.  vAMRL  Report  Mo.  AMRL-TR-7 3- lie , 
page  39,  publication  UNCLASSIFIED.) 


L i „ ... __ ljA 


Type  of 
OuCDUC 


IC«n  ID 


•r«eor 


□•scrip 


Scads 


Run  Si 


Run  number; 
das  dura- 
don. 

Opsr.  no.; 
scrsss 
threshold; 
speed  fac- 
tor. 

No.  of  ic- 
•radons; 
no.  success- 
ful; Z suc- 
cessful; 
av.  time 
per  Itera- 
tion; av. 
time  over- 
run; av. 
stress; 
av. , max. , 
min . and 
final 
Goal  As- 
piration; 
av.  wait- 
ing time. 

Also,  for 
subcasks: 
no.  failed; 
no.  ignored; 
no.  on  which 
peak  stress 
occurred; 
time  spent 
on  failed 
subtasks; 
etc. 


un  no. , 
iteration 


ame  as  run 
summary. 


Subtask  no.  & 
type;  essen- 
tiality. 

Operator  no. 


Run  number. 


Operator  no. 


ask  outcome;  Subcask  outcome 


total  time 
used;  time 
overrun; 
total  wait- 
ing time; 
stress-peak 
& final; 
goal  aspir- 
ation-max. , 
min . and 
final. 


subcask  time; 
waiting  time; 
stress;  cohes- 
iveness. 


Average  time 
used;  stress- 
peak  & final; 
prob.  of 
success;  wait- 
ing time;  sub- 
tasks Ignored 
& failed. 
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The  PSM  has  the  automated  capability  of  procassing  flight  Simula- 
tioa  data.  Hence,  aarly  predictions  can  ba  validated  by  actual  tasting.10 

The  relationship  between  the  PSM  and  other  elements  of  the  overall 
evaluation  technique  is  shown  in  Fig.  2,  adapted  from  a recent  paper.11 

The  use  of  the  model  requires  a number  of  sequential  actions  by 
the  user.  They  include: 

1.  Preparation  of  scenarios 

2.  Description  of  the  crew  station(s) , including  controls 

and  displays  and  their  locations. 

3.  Preparation  of  Operational  Sequence  Diagrams  (OSD's). 

4.  Description  of  each  operator  task  in  a standard  format. 

Each  task  is  described  as  an  Informacion-Processing-Action 
(IPA)  sequence.  Provisions  for  skipping  elements  of  the 
sequence  make  it  possible  to  use  the  same  sequence  for  all 
kinds  of  tasks. 

5.  Task  execution  time  distributions  and  error  probabilities 

for  each  task  oust  be  generated. 

6.  "Running"  the  model  for  a large  number  of  Iterations 
(model  iteration  limit  is  100). 

7.  Reviewing  and  analyzing  the  statistical  data  outputs. 

The  same  procedure  can  be  used  with  different  cockpits,  task 
assignments,  etc.,  as  needed. 

Inputs 

Like  all  man/machine  system  models,  the  PSM  is  critically  depen- 
dent upon  the  input  data  which  describe  the  mission,  the  vehicle,  the 
operator,  and  the  operating  sequences  required.  Among  the  inputs 
required  are: 

1.  Descriptions  of  all  tasks  and  values  of  associated  time- 

related  mission  variables.  ("Tasks"  are  defined  as 
simple  actions  such  as  "scan  display"  or  "adjust  ccntrol".) 

2.  The  distribution  of  expected  time  durations  for  each  task 

element,  from  external  sources. 

3.  Interactions  among  tasks,  from  operational  task  sequence 

diagrams. 

4.  Likelihood  of  error  in  performing  each  task,  from  external 

sources. 

^°Flight  Dynamics  Laboratory.  Definition  Study  for  an  Advanced  Fighter 
Digital  Flight  Control  Svstem,  by  D.S.  Hocker,  I.G.  Pope,  G.R.  Smith, 
and  G.J.  Vetch,  Wright  Patterson  Air  Force  3ase,  Ohio,  FDL,  June  1975. 
(AFFDL  Report  No.  AFFDL-TR-75-59 , publication  UNCLASSIFIED.) 

^Carl  F.  Asiala.  "Advanced  Man/Machine  Evaluation  Techniques"  pre- 
sented at  the  American  Defense  Preparedness  Assoc.,  Avionics  Section, 
Fail  Symposium,  Huntsville,  Ala.,  12-13  November  1975. 


5.  Criticality  and  difficulty  of  aach  task. 

6.  Locations  of  all  controls  and  displays. 

7.  Tima  panaltlas  associatad  with  task  sequences  batvaan  vldaly 

saparatad  controls/displays  (callad  "intar-zona  panaltias") 

8.  Dimans ions  of  oparator  visual  fiald,  including  af facts  of 

visors,  ate. 

9.  High-G  af facts  on  oparator  functions,  in  tabular  form. 

10.  Equipmant  idantif ication  and  raliability. 

11.  Cognitlva  and  parcaptual  motor  skills  sssociatad  with 

aach  task. 


Tha  primary  outputs  of  tha  PSH  ara  chose  associated  with  oparator 
workload  and  mission-characteristic  statistics.  Specifically,  outputs 
include : 

1.  Pilot  workload  as  a function  of  time. 

2.  Workload  of  individual  pilot  "channels"  (a.g.,  "right  hand," 

"visual") . 

3.  Timeline  plots. 

4.  Equipmant  utilization  data. 

5.  Mission  success  probability. 

6.  G-load  on  pilot  as  a function  of  time. 

7.  Required  display  character  size  and  luminance,  as  a function 

of  time. 


VOUGHT  WORKLOAD  SIMULATION  PROGRAM  (VOUGHT-WSP) 

Tha  Voughe  Corporation  has  developed  and  is  using  a cockpit  work- 
load prediction  modal  which  was  initially  referred  to  as  a Work  Require' 
mant  Modal, 12  and  now  as  a Workload  Simulation  Program. 


'T.J.  Klein  and  W.3.  Cassidy.  "Relating  Operator  Capabilities  to 
System  Demands,"  presented  at  1972  Human  Factors  Society  Annual 
Meeting,  Los  Angeles,  CA. , October  17-19,  1972. 


Vought  Systems  Division,  LTV  Aerospace  Corp.  Pilot  Task  Analysis/ 
Task  Description  Report:  TA-7C  Aft  Cockpit  vs  A-7E  Carrier  Recovery 
by  T.J.  Klein  and  A. A.  Hall.  Dallas,  Texas,  LTV,  March  1975. 

( Report  No.  2-34220/ 5R-17025 , report  ^’CLASSIFIED) . 
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TABLE  3.  Breakdown  of  Cockpit  Evaluation  Process  (McDonnell  Model)* 


Step  In  Process 


1.  Define  tasks: 

1.1  Mission  description 


1.2  Cockpit  equipment 
description 


2.  Determine  basic  task 

performance  data; 

2.1  Time  required  for 

each  task 

2.2  Accuracy  and/or 

reliability  of 
task  performance. 

2.3  Sequence  and  prior- 

ities 

3.  Modify  performance 

data  per  conditions: 
3.1  Time  adjusted  for 
reach  distance 


3.2  Penalties  for  G-load 
3.2.1  Grayout  penalty 


How  Handled 


External  to  Model 


3.2.2  Time  penalty 


Detailed  task  list; 
time -based. 

Locations  of  C/D 
elements;  type  of 
C/D  elements  (see 
2.1  & 2.2). 


From  literature  or 
man-in-loop  simula- 
tion, by  D/C  type. 

From  literature  or 
man-in-loop  simula- 
tion, by  D/C  type. 

Assigned  criticality 
during  task  analysis. 


Penalties  obtained, 
from  lit.  or  sim. , 
by  reach  distance. 


G vs  flight  conditions; 
grayout  threshold  - 
from  outside  sources. 


Time  penalty  vs  G,  by 
cask,  from  outside 
sources. 


Internal 


Available  time 
stored,  by  phase, 
as  a distribution. 

Location  "zone" 
stored  for  each 
piece  of  equip- 
ment. 


Stored  distribu- 
tion, by  task. 

Stored  as  human 
probability 
learning  curve. 

Criticality  code 
attached  to  each 
stored  task. 


Time  penalty  com- 
puted, using  lo- 
cation (from 
1.2)  and  stored 
data. 

Compute  G-load 
dynamically; 
consult  stored 
data  & delay  if 
G>G  threshold. 

Compute  G-load 
dynamically; 
look  up  scored 
cime  penalty; 
apply  penalty 
to  basic  cask 
time. 


16 


NWC  TP  6020 


TABLE  3.  (Continued.) 


Step  In  Process 


4.  Determine  operator 
task  load: 

4.1  Task  load  by  task 
and  by  sensor/ 
effector 


4.2  Total  operator  load 


5.  Determine  system  per- 
formance: 

5.1  Equipment  utilization 


.2  Control/display 
location  effects 
(link  analysis) 


5.3  Displayed  data 
req ' ts 


How  Handled 


External  to  Model 


Stored  data  on  visual 
capability  under  G- 
load  and  other  en- 
vironments. 


Internal 


Select  basic  task 
time  by  Monte 
Carlo  from 
stored  data 

(2.1) ;  modify  by 
penalties  (3) ; 
compare  with 
available  time 

(1.1)  selected 
by  Monte  Carlo; 
compute  Z task 
load  for  each 
member. 

Sum  element  loads 
from  4.1;  print 
out  as  function 
of  mission 
phase.  (Mote 
that  several 
tasks  may  occur 
at  same  time; 
load  may  be 
>100Z). 


Output  frequency 
of  use;  percent 
utilization  by 
mission  phase. 

Output  all  "look 
links"  and 
"reach  links" 
between  C/D 
elements. 

Generate  (from 
stored  data) 
symbol  size  4 
luminance  mini- 
mums,  by  phase. 


I 

1 

1 

1 


i. 
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TABLE  3.  (Continued.) 


Step  in  Process 


Hov  Handl 

Extsrnsl  to  Modsl 


Intsrnsl 


5.4 


Accuracy/ rsllabllicy 


Input  human  rallabilicy 
by  cask. 


Modified  by  input 
"learning  curve” 
Output  probabil- 
ity of  success. 


A brief  description  of  the  program  is  found  in  an  Internal  document 
provided  to  the  writer  by  J.  E.  Burke  of  the  Vought  Corporation: 

"The  workload  simulation  program  is  a time-based  computerized 
mathematical  modeling  technique  for  predicting  workload 
levels  imposed  upon  the  human  by  mission  and  system  para- 
meters. The  model  provides  discrete  task  (switch-throwing, 
etc.)  and  continuous-control  task  (stick/ throttle  modulation) 
difficulty  level  predictions  over  the  spectrum  of  mission 
flight  segments. 


"General  capabilities  of  the  model  include: 

. crew  sizing  and  man-machine  function  balancing 

. prediction  of  the  impact  on  crew  workload  demands 
of  variations  in  aircraft  design  parameters, 
control/display  arrangement,  avionics  equipment, 
flight  path  deviation  tolerances,  system  inte- 
gration logic  (moding),  and  mission  and  environ- 
mental considerations. 

. identification  of  potential  task  overload  points 
for  man-machine  system  trade-off  consideration 

. assessment  of  the  Impact  on  crew  workload  demands 
that  can  be  expected  to  result  from  assigned 
system  functions 

. predicting  work  response  levels  from  known  systems 
demands 

. relating  cracking  accuracy  to  system  workload 
demands 

. combining  cracking  and  procedural  workload 
components" 

General  Structure 


The  model  is  composed  of  two  primary  sub-models  - one  for  discrete 
control  casks,  the  ocher  for  continuous  control.  Input  data  flow  into 
one  or  Che  other  of  the  submodels.  The  oucpucs  from  the  two  submodels 
are  merged,  after  separata  "task  load  indices"  have  been  computed, 
yielding  a total  workload  by  mission  phase,  as  required  by  Mil-H--o355. 


i8 


i 
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' 

The  overall  organization  of  the  model  is  shown  in  Fig.  3 and  wee  adapted 
from  the  internal  Vought  Corporation  report  cited  in  the  preceding 
paragraph . 

| 

Inputs 

Aa  shown  in  Fig.  3,  inputs  are  broken  into  three  types. 

Misaion/Taak  Variables.  These  inputs  provide  the  characteristics 
of  the  mission  (e.g.,  segment  sequence,  flight  path  tolerances,  time 
envelopes)  required  to  call  up  all  the  casks  Involved  in  the  mission. 
Priorities  (on  a five-point  scale)  are  assigned  to  each  Cask.  For 
each  cockpit  cask,  a task  duration  time  (TDT)  estimate,  based  on  empirical 
data,  is  input. 

Cockpit  Configuration  Variables.  These  inputs  include  cockpit  geom- 
etry and  control  and  display  locations  in  Che  cockpit.  Tocal  task  times 
required  are  computed  and  include  Che  TDT  estimate  and  the  additional 
effects  of  control  and  display  locations. 

■ 

Controlled  Svstem  Variables.  These  inputs  relate  to  aerodynamic 
parameters  of  Che  aircraft,  and  are  used  to  compute  the  race  at  which 
deviations  from  flight  path  will  occur.  These  deviation  races,  together 
with  the  flight  path  tolerances  mentioned  above,  define  Che  cask  loading 

in  continuous  control  casks. 

[ i 

Outputs 

Figure  3 indicates  only  some  of  Che  direct  outputs  of  Che  model. 
However,  the  output  includes  additional  information  which  is  useful  in 
evaluating  specific  crew  station  designs.  For  discrete  tasks,  the  mean 
cask  total  performance  time  required  is  divided  by  the  available  time 
and  sunaed  on  an  earliasc-time-of-iniciation,  latest-time-of-compleclon 
basis  to  generate  "worst-case”  discrete  task  time  stress  (T-S)  values. 

An  additional  output  available  for  evaluation  is  "time-wichin- 
tolerance"  for  all  three  flight  path  channels  for  continuous  control 
casks  - a measure  of  Cracking  accuracy  and  qualicy  of  mission  performance. 

The  summed  (discrete  plus  continuous  task)  workload  level  for  each 
time  segment  is  compared  with  a "workload  limit"  (which  is  empirically 
determined,  and  varies . as  a function  of  the  percent  mix  of  continuous/ 
discrete  tasks  during  that  segment)  to  determine  whether  a potential 
overload  exists.  A second-by-second  plot  of  workload  demand  is  Che 
output. 

Simplified  Simulation  Flow 

A typical  cockpit  evaluation  sequence  for  a discrete  task  is  shown 
in  Table  4.  It  shows  how  various  cask,  mission  and  operator  factors 
are  handled,  either  internal  to  the  model  or  externally. 


•3 


TABLE  4.  Breakdown  of 


Seep  is  Process 


1.  Define  Tasks 


1.1  Mission  description 


1.2  Cockpit  equip, 
description 

2.  Determine  discrete  task 


erformance 


2.1  Time  required  to 
perform  each  task 

2.2  Time  to  reach  or 

look  at  next  task 


of  task  performance 


2.4  Sequence  & Prior- 
ities 


3.  Modify  performance  data 


er  conditions 


3.1  Tasks  delayed  by 
"overload"  (see 


How  Handled 

External  to  model 

Internal 

Time-based  detailed 

Store  sequence  & times 

task  list  per  mission 

required  per  task. 

profile  segment. 

Compute  segment  time 

envelopes . 

Locations  of  C/D 

Store,  by  equipment 

elements ; types  of 

x,  y,  z locations. 

C/D  elements. 

Median  value,  from 

Stored  data  by  task. 

test  data  or  esti- 

mates. 

Test  data  used  to 

Algorithm  for  "reach 

develop  algorithms. 

time"  or  "look  time" 

as  function  of  dis- 

tance . 

Discrete-no t-com- 

Discrete-not  com- 

puted. 

puted. 

Continuous-deter- 

Continuous -genera  tes 

mine  control 

performance  data 

tolerances  by 

from  task  descrip- 

mission profile 

tion,  control  toler- 

segment. 

ances,  and  empirically 

generated  equations.  \ 

Assign  criticality 

Interlace  tasks , 

by  task  & phase. 

using  frequency  & 

Frequency 

criticality,  by 

of  monitoring 

Monte  Carlo  method. 

determined  by  error 

nulling  frequency 

requirements . 

Establish  critical 

Delay  least  critical 

task  load  empiri- 

task & recompute 

cally.  Assign 

task  load. 

priorities  (see 

I 

2.4). 

Sup  in  Process 


ow  Handled 


External  to  model 


Internal 


3.2  Penalty  for  hi-G,  or  Determine  G-related 
any  other  degrading  decrements;  tabu- 
environmental  late, 

condition 


Test  for  G & apply 
decrement  if  needed. 


3.3  Modify  for  night, 
weather,  defenses 


4.  Determine  ooerator 


task  load 


4.1  Manual  'time- 

stress"  (T-S) 
for  discrete 

tasks 

4.2  Sensory  "time- 

stress"  for 
discrete  casks 

4.3  Combined  "time- 

stress"  (dis- 
crete plus  con- 
tinuous) by  seg- 
ment of  mission 
profile  * 

4.4  System  operator 

workload  demand 


5.  Determine  svstem  per 


romance 


3.1  Accuracy/ probab 11 
ity  of  success 


5.2  Concrol/display 
location  effects 
(link  analysis) 


establish  new  casks;  Scored  data  changed, 
criticalities,  decre- 
ments, tolerances. 


Available  time  for 
each  cask  deter- 
mined (see  1.1  & 
2.4). 

Same  as  above. 


Same  as  above. 


jReconanended  "T-S" 
design  limit 
determined 
empirically. 


See  2.3  and  2.4. 


See  1.2  4 2.2. 


Required  time  (2.1  & 
2.2)  compared  with 
available  time , by 
cask. 

Same  as  above . 


Discrete  "T-S"  com- 
bined with  contin- 
uous task  "T-S", 
once  per  second. 


"T-S"  computed,  com- 
pared with  "T-S" 
recommended . 


Computes  probability 
of  remaining  in 
tolerance  in  all 
channels . 

Computes  control-to- 
control  and  displav- 
to-displav  link 
values . 


T-S  for  "continuous"  tasks  is  determined  by  comparing  the  (empirical) 
time  required  to  correct  a deviation  with  the  error  correction  race 
imposed  (see  2.3  and  2.4) . 
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NAVAL  AIR  DEVELOPMENT  CENTER  HOS  MODEL 

The  Naval  Air  Development  Center  (NADC)  at  Warminster,  Pa.,  has 
directed  two  large-scale  efforts  In  operator/cockpit  modeling.  One  of 
these  efforts  (known  as  "CAFES")  was  performed  under  contract  by  Boeing 
Aircraft  Corp.,  and  Is  discussed  separately.  The  other  major  effort 
(known  aa  the  Human  Operator  Simulator,  or  "HOS")  has  been  performed  by 
Analytics,  Inc.,  as  well  as  by  personnel  at  NADC  and.  In  earlier  years, 
at  the  Naval  Missile  Center  (NMC) , Pt.  Mugu,  California.  Commander 
R.  J.  Wherry  has  directed  the  HOS  work  from  its  inception  at  NMC. 
Funding  for  the  program  has  come  from  the  Naval  Air  Systems  Command. 

The  HOS  program.  Including  the  Human  Operator  PROCedures  (HOPROC) 
Language,  the  HOPROC  Assembler  and  Loader  (HAL),  and  the  Human  Operator 
Data  Analyzer/Collator  (RODAC) , is  an  ambitious  attempt  to  model  the 
human  operator  in  all  his  complexity  from  micro-models  at  a detailed 
behavior  level.  A recent  paper  by  R.  J.  Wherry,  Jr.,  che  originator  of 
the  HOS  program,  states 

"The  Human  Operator  Simulator  (HOS)  digital  computer  program 
developed  over  the  past  seven  years,  is  capable  of  simulating 
the  performance  of  a goal-oriented,  adaptive,  trained  human 
operator  in  a complex  weapon  system  down  to  che  level  of  hand 
reaches,  control  device  manipulations , eye  shifts,  absorptions 
of  visual  information,  and  internal  information  processing  and 
decision  making. 

The  combined  effects  of  an  operator's  role,  the  displays  and 
controls  he  would  be  given  and  their  layout,  and  che  various 
situations  with  which  he  would  be  confronted  are  obviously 
complex.  Existing  human  performance  prediction  techniques 
do  not  handle  such  complex,  interactive  situations  in  which 
there  are  hundreds  of  variables  chat  could  effect  his  per- 
formance. It  is  axiomatic  chat  such  complex  situations  must 
be  created  and  exercised  to  find  out  what  will  really  happen 
to  the  operator." 

The  objective  of  this  elaborate  modeling  effort  is  to  provide  an 
acceptable,  valid  substitute  for  "expert  opinion"  in  the  design  and 
evaluation  of  complex  systems.  Again,  in  Wherry's  words:^ 


^R.J.  Wherry,  Jr.  "The  Human  Operator  Simulator  - HOS"  in  Monitoring 
3ehavior  and  Supervisory  Control,  ed.  by  T.3.  Sheridan  and 
G.  Johannsen.  NATO  Conference  Series,  III  Human  Factors,  Vol.  1. 
New  York,  Plenum  Press,  1976.  p.  283 

^R.J.  Wherry,  Jr.  "The  Human  Operator  Simulator  - HOS"  in  Monitoring 
3ehavior  and  Supervisory  Control,  ed.  by  T.B.  Sheridan  and 
G.  Johannsen.  NATO  Conference  Series,  III  Human  Factors,  Vol.  1. 
New  York,  Plenum  Press,  1976.  p.  284. 
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"Tha  need  co  create  Cha  situation  and  to  collact  human  per- 
formance data  from  tha  craatad  situation  led  to  tha  con- 
clusion that  simulated  man- in- t ha-loop , dynamic  simulation 
would  ba  an  accaptabla  substitute  if  human  operator  behavior 
in  a complex  craw  station  could  ba  simulated  with  sufficient 
accuracy  and  detail.  With  the  advent  of  high  spaed,  digital 
computers,  a sufficiently  sophisticated  modal  of  how  a human 
operator  behaves  became  feasible.  HOS  is  an  attempt  at  such 
a model." 

The  attainment  of  the  HOS  objective  is  critically  dependent  upon 
the  ability  of  the  user  to  construct  or  acquire  adequate  "micro-models" 
from  which  to  build  up  the  larger  procedures  involved  in  system  opera- 
tion. These  micro-models  must  represent  such  "micro-behavior"  elements 
as  reach,  grasp,  twist  a control,  read  a digit,  remember  a digit.  If 
the  assemblage  of  micro-models  is  to  reflect  accurately  the  effects  of 
such  variables  as  control  knob  size  and  spacing,  training  level,  tem- 
perature stress,  etc.,  then  all  micro-models  must  either  (a)  be  accur- 
ately sensitive  to  those  variables,  or  (b)  represent  actions  which  are 
unaffected  by  them. 

One  major  difference  between  the  HOS  approach  and  the  two  preced- 
ing models  is  the  use  of  more  finely  granulated  behavior  elements  des- 
cribed above,  rather  than  "1-P-A"  units  (McDonnell)  or  "tasks"  (Vought) 
The  HOS  model  also  postulates  a greater  variety  of  responses  to  an 
event  (e.g.,  "recall  present  setting"  or  "look  at  present  setting")  and 
more  detailed  record-keeping  on  position  of  body  parts  (e.g.,  HOS  main- 
tains 3 positions  for  each  hand  - "present,"  "preferred,"  and  "relax- 
ed”). The  entire  HOS  concept  envisions  a much  more  fluid,  adaptive 
operator  model  than  other  models  in  use. 

General  Structure 

A decision  must  be  made,  before  describing  the  structure  of  HOS, 
whether  to  describe  the  conceptual  HOS  or  the  existing,  operating 
version.  While  the  concept  of  HOS  is  most  interesting  and  valuable  as 
an  outline  of  a sophisticated  approach  to  operator  modeling,  the  empha- 
sis in  this  review  on  "available"  models  seems  to  require  that  the  op- 
erating versions  are  the  ones  to  be  described  here.  It  is  recommended 
that  the  Interested  reader  obtain  the  extensive  documentation  on  HOS 
before  embarking  on  any  substantial  effort  to  develop  new  models  or 
modal  elements. 16-19 


^Analytics.  The  Human  Operator  Simulator,  Volume  I,  Introduction  and 
Overview,  by  M.I.  Streib.  Willow  Grove,  PA.,  August  1975.  (Analytic 
Technical  Report  1117-1,  publication  UNCLASSIFIED . ) 

^'Analytics.  The  Human  Operator  Simulator,  HOS  User 3 Guide,  bv  M.I. 
Streib.  Willow  Grove,  PA.,  October  19’5.  (Analytics  Technical 
Report  11S1-A,  publication  UNCLASSIFIED . ) 
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The  basic  structure  of  the  HOS  system  in  simulating  an  aircraft 
mission  would  be  as  shown  in  Fig.  4,  from  Analytics'  Introduction  and 
Overview  volume  (referenced  above).  Unlike  the  other  models,  the  simu- 
lation is  driven  by  a procedure-based  "Mission,"  which  is  a branching 
series  of  tasks  or  steps.  The  impact  of  the  mission  on  the  operator  is 
through  one  of  two  types  of  "change"  signals:  (1)  mission-specified 
changes  (such  as  turn  points)  and  (2)  corrective  changes  (in  response 
to  exogenous  events,  or  to  a displayed  quantity  drifting  outside  pre- 
scribed limits)  . 

When  a change  (or  "ALTER"  in  HOS  terms)  is  specified  from  the 
mission,  the  QOS  model  first  checks  whether  the  "new"  value  already 
exists  as  a result  of  some  earlier  action  (see  "Recall"  in  Figure  4) . 

If  not  ("Decision"),  it  calls  up  a "change  value"  subroutine  (using 
"Anatomy  Mover").  If  the  value  is  already  in  effect,  the  operator 
"memory"  (a  function  of  learning  level  and  elapsed  time,  or  "0-State") 
is  tested  to  determine  whether  the  value  must  be  interrogated  or  not. 
This  alternative  clearly  adds  to  the  flexibility  of  operator  response; 
validity  depends,  of  course,  upon  the  algorithms  and  input  quantities. 

Parallel  actions  can  occur,  if  they  are  in  different  "faculties" 
(e.g.,  body  movement  and  memory  recall). 

Variability  of  performance  is  to  be  handled  in  HOS  by  time-varying 
"0-states"  which  presumably  reflect  accurately  the  effects  of  practice, 
stress,  etc.,  rather  than  by  Monte  Carlo  methods.  However,  in  one 
closely-documented  simulation  application,  it  is  stated  chat  "chough 
we  hope  evencually  Co  develop  a decailed  cheory  of  how  display-reading 
performance  is  influenced  by  cralning,  fadgue,  accencion  and  autoraati- 

city Che  quantitative  version  of  Chat  cheory  is  noc  co  be  achieved 

in  the  near  future. "20  The  operator  can  "choose"  Co  omlc  seeps  in  the 
procedures  if  they  are  noc  required  by  che  current  system  status.  He 
can  also  delay  or  omit  seeps  when  che  crlcicalieies  of  low  priority 
tasks  remain  below  ocher  criticalities.  Considerable  flexibility  is 
available  in  che  procedures  coding  co  dynamically  change  criticalities 
as  a function  of  "current"  events  during  simulation. 


Analy t ic s . The  Human  Operator  Simulator.  Volume  VI,  Simulation 
Descriptions  . by  M.I.  Streib.  Willow  Grove,  PA.,  December  1975. 
(Analytics  Technical  Report  1181-B,  publication  UNCLASSIFIED.) 

^Analytics.  The  Human  Qperacor  Simulator,  HAL  Programmer's  Guide,  by 
M.I.  Streib.  Willow  Grove,  PA.,  March  1976.  (Analytics  Technical 
Report  1181-C,  publication  UNCLASSIFIED.) 

^Analytics.  Development  of  a Quantitative  ni  «n  1 .IV  3 rl  i*  r»  o Mori  a 1 


Analytics.  Development  of  a Quantitative  Display  Reading  Model  for 
HOS , by  M.I.  Streib.  Willow  Grove,  PA.,  May  1975.  (Analytics 
Technical  Report  1117-F,  p.  4-3,  publication  UNCLASSIFIED.') 
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Program  Modules 

Tha  HOS  taodal  itself  (aa  diatincc  from  cha  inputs,  outputs , ate.) 
is  mada  up  of  four  principal  modulas. 

1.  Tha  Dacodar.  This  modula  racaivas  instructions  (from  tha 
mlssion-basad  inputs)  and  axarcisas  a sarlas  of  queries, 
to  datarmlna  tha  status  of  tha  ralavant  aquipmant  and 
oparator  alamants,  follovad  by  a dacision  aa  to  tha  bast 
saquanca  of  staps  to  ba  follovad. 

2.  Tha  Multlplaxor.  This  modula  maintains  prioritiaa  of 
possibla  procaduras,  and  saquancas  tha  instructions  sant 
to  tha  Dacodar,  Including  intarrupts  if  raquirad.  If  no 
action  is  raquirad,  tha  multlplaxor  will  althar  return 
tha  system  to  an  axistlng  procedure,  or  enter  a "relax" 
mode  (in  which  operator  recovery  from  fatigue  could  ba 
modeled,  although  it  is  not  presently  represented.) 

3.  Tha  Estimator.  This  modula  models  tha  sensing  and 
memory  aspects  of  the  operator  actions.  For  example, 
whenever  an  operator  procedure  calls  for  "reading"  a 
device,  tha  estimator  begins  a sequential  proce**' : 

(1)  If  tha  oparator  is  already  in  "contact"  with  the 
device,  ha  is  assumed  to  "read"  or  "absorb"  its  value; 

(2)  if  ha  is  not  in  contact,  recall  of  the  value  is 
attempted;  (3)  if  recall  cannot  be  accomplished,  he 
must  move  tha  necessary  element  to  the  equipment;  and 
(4)  then  absorbs  the  reading.  Time  charges  are 
assessed  for  all  actions. 

4.  The  Banker.  This  module  accumulates  time  charges  (as 
wall  as  potentially  keeping  track  of  changes  in  0-state 
due  to  fatigue,  etc.). 


Inputs 

Inputs  to  HOS  can  occur  In  two  ways.  The  HOPROC  procedures  langu- 
age is  used  to  provide  five  classes  of  inputs,  as  described  brleilv 
below. 21 

1.  "Title  declarations."  These  inputs  identify  the  controls, 
displays,  symbols,  and  settings  associated  with  the  cock- 
pit Interface,  and  the  "0-states"  of  the  operator^). 

2.  "Operator  functions."  These  Inputs  are  in  equation  form 
and  describe  the  mental  computations  available  to  the 

“*Analvtics.  The  Human  Operator  Simulato- , Vol.  I,  Introduction  and 
Overview,  by  M.I.  dtreib.  Willow  Grove , ?.\.  , August  WT3.  ^Ana- 
1 vtic s Technical  Report  UlT-l,  pp.  1-1  to  ■>?,  puolication 
UNCLASSIFIED.) 
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operator.  The  aquations  may  incorporata  tha  valuas  of 
othar  davlcas  or  functions. 

3.  "Hardware  functions."  These  Inputs  are  In  aquation 
form,  similar  to  tha  "operator  functions." 

4.  "Operator  procedures."  The  user  seldom  accesses  the  basic 
"microactivities";  these  are  ordinarily  called  and  executed 
by  a H0PR0C  instruction  (like  ALTER).  The  instruction  is 
automatically  compiled,  and  a microactivity  sequence  is 
generated  based  on  internal  rules.  All  procedures,  whether 
system- inherent  or  user-constructed,  are  treated  as  units 
for  execution  purposes;  they  function  like  subroutines. 

5.  "Hardware  procedures."  These  are  parallel  in  form  to 
the  "operator  procedures." 

In  addition  to  the  above  HOPROC -moderated  inputs,  the  HOS  receives 
several  types  of  direct  inputs: 

1.  Initial  states  of  devices  and  functions,  and  initial 
"0-statcs." 

2.  Locations  of  all  devices. 

3.  Initial  "hab"  strength  (a  measure  of  how  well  an 
operator  function  has  been  learned,  and  how  long 
since  it  was  reinforced). 

4.  The  amount  of  tine  required  for  a simple  interrogation 
of  a display. 

3.  The  accuracy  associated  with  an  estimate  of  expected 
value  of  a display  or  control  setting. 

6.  Indicators  associated  with  equipment,  specifying  the 
type  of  equipment,  and  such  parameters  as  the  time 
required  to  make  an  adjustment. 

Outputs 

The  outputs  available  from  HOS  are  printouts  of  all  instruction 
executions  and  event  times.  The  compilation  of  this  raw  output  mater- 
ial into  useful  quantities  (such  as  loading,  number  of  events  by  class, 
timelines,  etc.)  is  performed  by  a separate  module  called  HODAC  (Human 
Operator  Data  Analyzer/Collator) . HODAC  can  generate  seven  types  of 
reports: 

1.  Timeline  Analysis  - a listing  of  activities,  by  body 
part,  on  a time  base. 

2.  Device-by-3ody  Part  - cumulative  statistics  on  the  amount 
of  time  spent  by  each  body  part,  in  each  type  of  activitv, 
on  each  device. 
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3.  Device-by-Usage  - cumulative  statistics  on  number  of 
actions  and  tlma  spent  in  each  type  of  action. 

4.  Device-by-Procedure  - same  as  above,  but  keyed  to 
task  elements  (e.g.,  "remove  the  symbol"). 

3.  Procedural  Analysis  - cumulative  statistics  as  above, 
but  keyed  to  significant  procedure  types  (e.g., 

"deciding") . 

6.  HODAC  Label  Analysis  - cumulative  statistics  on  the 
occurrences  of  HODAC  "statement  labels”  by  mission 
procedures. 

7.  Link  Analysis  - cumulative  statistics  on  movements  by 
body  part,  from  device  to  device  in  the  cockpit. 

Typical  Cockpit  Evaluation 

The  HOS  modal,  in  its  present  (1976)  state,  is  not  sufficiently 
complete  to  permit  a whole  mission  cockpit/workload  evaluation  process 
which  is  directly  comparable  to  the  preceding  models.  Table  S presents 
those  elements  available  from  the  documentation,  in  as  nearly  compar- 
able form  as  feasible.  It  should  be  kept  in  mind,  however,  that  the 
HOS  model  has  not  been  exercised  in  a whole-mission,  whole  cockpit  sim- 
ulation as  have  the  preceding  models.  The  most  extensive  documented 
airborne  mission  simulations  are  two  segments  of  the  Air  Tactical 
Officer  duties  in  the  LAMPS  helicopter  system.--  The  two  mission 
elements  are: 

1.  Keyset  entry  and  retrieval.  This  task  involves  a key- 
board and  an  alphanumeric  display.  The  simulation 
involves  modification  of  settings  in  a stores-control 
system. 

2.  Fly-to  Contact.  This  task  involves  a cursor  control 
stick  and  two  associated  switches,  and  a map-type 
display  which  shows  contact  locations  and  the  location 
of  the  cursor.  The  simulation  involves  erasing  three 
existing  contacts  on  the  display. 


BOEING  "COMPUTER  AIDED  FUNCTION-ALLOCATION  SYSTEM"  (CAFES) 

CAFES  is,  as  the  name  implies,  a system  for  computer-aided  cockpit 
analysis  and  design.  It  is  made  up  of  several  modules,  some  of  which 
are  "models"  in  the  sense  that  we  have  used  the  term  before.  Taken 
collectively,  CAFES  is  Intended  to  include  essentially  all  che  features 


"“Analytics.  The  Human  Operator  Simulator,  Volume  VI,  Simulation 
Descriptions,  by  M.  I.  Streib.  Willow  Grove,  PA.,  December  19*5. 
(Analytics  Technical  Report  1181-3,  publication  UNCLASSIFIED). 
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TABLE  S.  Elements  of  Cockpit  Evaluation  Process  (HOS  Modal). 


Stap  In  Procass 

How  Hand) 

Led 

External  to  Model 

Internal 

1.  Daflna  tasks: 

1.1  Mission  description 

Event-based,  branch- 

Stored  in  "Mission"; 

ing,  detailed  task 
list. 

fed  to  "Decoder". 

1.2  Cockpit  aqulpoant 

Locations  of  all 
equipment. 

Stored . 

2.  Determine  task  per- 
formance : 

2.1  Time  required  to 

Handled  both  externall; 

r and  internally. 

perform  task 

Most  time  elements  are 

determined  by 

elements,  or  to 

internal  algorithms  while  other  estimates 

transition  from 

are  input  parameters  to  provide  for 

task-to-task 

variability. 

2.2  Accuracy/reliabil- 

Required  accuracy  is 

Sequence  can  be 

ity  of  operator 

input;  system  will 

changed  by  the 

ac  tion 

manipulate  absorp- 

operator  or  dynam- 

tions  and  concrol  the 

ically  by  the  pro- 

inputs  to  try  to 

cedure,  according 

achieve  this  accuracy 

to  the  criticality 

level . 

value . 

2.3  Accuracy/reliabil- 

Establish  memory 

Operator  memory  of 

ity  of  operator 

functions  in  terms 

display/control 

memory 

of  practice,  elapsed 

states  is  fallible. 

time,  0-state,  ecc. 

and  is  time- 
variable. 

2.4  Sequence  and  prl- 

Criticality  assigned 

Criticality  modified 

oritles 

co  casks. 

as  function  of 
mission  state,  dis- 
played value,  and 
time  since  last  read- 
ing. Sequence  varies 
with  criticality,  and 
criticality  can  vary 
with  sequence. 
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TABLE  5.  (Continued.) 


How  handled 

External  to  Model 

Internal 

3.  Modify  performance 
per  conditions: 

3.1  Effects  of  stress, 
fatigue,  etc. 

Establish  "0-state" 
theory*  and 
functions . 

Algorithms  planned.* 

3.2  Effects  of  weath- 

er,, defenses,  etc. 

3.3  Effects  of  pos- 

ture, anthropo- 
metries 

Defined  by  "Mission" 
inputs . 

Anthropometry  input 
to  ANATOMY  MOVE- 
MENT model. 

No  change  in 
operation. 

Keeps  track  of  limb 
positions;  computes 
times  for  required 
moves . 

4.  Determine  Operator 

Task  Load 

4.1  Overall  activity 

level 

4.2  Load  by  body 

element 

4.3  Body  movement 

load 

4.4  Effects  of  over- 

load 

Output  "Timeline 
Analysis"  report. 

Output  "Davice-by- 
Body  Part”  report. 

Output  "Link  Analy- 
sis" report. 

"Overload"  not  per- 
mitted; unneeded 
tasks  may  be 
delayed/omitted . 

3.  Determine  Svstem 

Perfo rmance: 

3.1  Effects  of  cock- 
pit arrangement 

Select  alternative 
arrangements . 

Output  "Link  Analy- 
sis" report. 

Performance  algorithms,  internal  to  model,  are  planned  but  not  avail- 
able. External  factors  are  managed  by  defining  alternative  procedures 
f she  operator  to  shift  to  when  his  performance  degrades  or  Improves. 
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used  in  any  of  Che  models  previously  described.  However,  noe  all 
modules  are  compleced  and  documented  at  this  writing. 

Completed  modules  which  have  characteristics  similar  to  the  other 
models  described  in  this  report  are  the  Function  Allocation  Model  (FAM) , 
which  is  made  up  of  two,  almost  Independent  sections,  and  the  Workload 
Assessment  Model  (WAM) . The  Computer  Aided  Design  (CAD)  module  is 
entirely  geometrical,  but  is  not  a man/machlne  model.  Reach  envelopes 
for  CAD  have  to  be  generated  by  the  user  and  are  a required  input  for 
the  CAD  Reach  Analysis  Module.  CAD  also  has  software  modules  which 
perform  a vision  analysis  and  escape  Interference  analysis.  The  entire 
CAFES  system  (except  HOS)  is  being  developed  by  Boeing  Aircraft  Co., 
under  contract  to  the  Naval  Air  Development  Center.  The  following 
descriptions  are  adapted  from  a Boeing  report. 23 

FAM-1  (Mission  Evaluation) 

The  Mission  Evaluator  Module  is  directed  toward  the  limited  objec- 
tive of  computing  a mission  success  probability  for  each  of  a number  of 
alternative  task  allocations,  providing  a rank-ordering  of  the  alterna- 
tives on  this  single  criterion.  It  also  computes  success  probabilities 
for  specific  mission  objectives  within  the  mission. 

Inputs.  The  input  data  required  by  the  Mission  Evaluator  includes 
(1)  a mission  scenario  with  cask  list,  and  times  for  occurrence  of 
tasks,  (2)  a sec  of  mission  objectives  (e.g.,  target  acquisition, 
threat  suppression)  with  alternate  task  sequences  leading  to  each,  (3) 
complete  operator  data  for  each  cask  (e.g.,  average  time  required, 
error  race,  and  a task-load  rating),  and  (4)  trial  allocations  of  casks 
to  crew  members  and/or  automated  equipment.  The  task-load  rating  com- 
bines racings  of  precision,  concentration,  reliability,  continuity 
(non-interruptability) , and  priority/cricicality,  in  simple,  additive 
fashion. 

Process  and  output.  The  objective  of  the  process  is  to  arrive  at 
an  overall  "mission  success"  value  which  is  sensitive  to  task  alloca- 
tion. The  basic  data  for  the  "operator  reliability"  element  is  a 
"reliability-vs  cask  execution  time"  function.  A multi-step  procedure 
(much  of  it  external  to  the  model)  for  arriving  at  operator  reliability 
as  a function  of  task  loading  is  described  in  the  Boeing  report.  "Task 
reliability"  includes  both  the  operator  estimate  and  an  input  value  for 
equipment  reliability. 

Overall  "mission  success"  estimates  are  derived  in  several  forms. 
Most  simply,  overall  mission  success  is  computed  as  the  product  of  the 
cask  reliabilities.  A "modified  mission  success"  figure  uses  weights, 

23 

Boeing  Aircraft  Co.  Computer-Aided  Function  - Allocation  Evaluation 

System  - Volume  2.  by  A.F.  Anderson,  K.S.  Renshaw,  and  D.C.  Whitmore. 

Seattle,  Washington,  3AC,  December  1974. 
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related  to  importance  of  Che  casks,  co  modify  ch«  "cask  reliabilicy” 
asciaacas.  Saparaca  probabilicias  for  individual  mission  objectives 
can  also  ba  provided.  J 


FAM- 2 ( Procedure  Generator) 

The  Procedure  Ganaracor  module  genaraces  operational  procedures 
aa<iuancas)  and  compucas  preliminary  workload  statistics. 

Inputs . In  addition  to  Che  FAM-1  inputs,  the  Procedure  Generator 
requires  a sat  of  rules  and  constraints  for  task  precedence.  These 
include  mandatory  casks,  prerequisite  tasks,  interruptability,  earliest/ 
latest  start  times.  An  important  added  factor  is  the  "RNO  Function" 
(Remaining  Number  of  Opportunities)  which  has  the  effect  of  increasing 
the  priority  of  a delayed  task  as  the  "latest  start  time"  draws  near. 

Process  and  Output.  The  basic  process  of  the  Procedure  Generator 
is  the  selection  of  tasks  in  a sequence  determined  by  the  rules  and 
constraints  (see  Inputs)  interacting  with  Che  dynamic  RNO  Function. 

The  primary  output  is  the  operational  procedure,  giving  task  sequence 
and  times.  In  addition,  statistical  data  on  top-level  workload  esti- 
mates (e.g.,  percent  of  time  each  operator  is  occupied)  are  output,  for 
each  trial  function  allocation. 

WAM  Module  (Workload  Assessment  Model) 

The  basic  purpose  of  the  WAM  (and  the  "Statistical  WAM,"  or  SWAM) 
is  to  provide  more  detailed  workload  data  (e.g.,  for  hands,  eyes)  as  a 
function  of  system  design  and  task  allocation  alternatives. 

inputs.  The  basic  input  to  WAM  is  a highly  detailed  task/cimeline. 
The  kind  or  data  required  for  FAM  is  used  as  a basis  for  the  input,  but 
much  more  detail  is  required.  The  mission  is  broken  into  variable  time 
segments  from  one  second  up  and  crew  activity  by  "channel"  (e.g.,  ayes, 
hand)  for  each  segment.  This  is  all  prepared  external  to  the  WAM; 
through  the  OMS,  some  FAM  output  is  directly  usable  as  WAM  input. 

Process  and  Output.  The  WAM  converts  the  input  data  into  percent 
workload,  by  channel,  by  segment.  These  workload  data  are  treated 
statistically,  with  the  results  put  out  as  tabulations  and/or  plots,  bv 
mission  phase.  Overload  conditions  are  flagged,  and  detailed  contribu- 
tions co  overload  identified.  A limited  cask  shifting  capability  is 
currently  included  in  the  WAM  software. 


Typical  Cockpit  Evaluation 

It  should  be  recalled  that  CAFFS  is  not  a model,  but  a framework 
for  a sec  of  models,  not  all  of  which  are  currently  functioning.  It 
is  not  feasible  co  produce  a sec  of  iterations  of  a mission  with  a 
single  computer  run,  as  might  be  done  with  the  first  three  models  dis- 
cussed. The  separate  submodels  (e.g.,  FAM- l and  -2,  WAM,  CAD)  are  run 
separately,  with  some  off-line  manipulation  of  data  between  runs. 


Consequently,  the  entries  in  Teble  6 ere  not  strictly  coopereble 
with  those  in  Tables  1,  3 and  4,  but  represent  a composite  of  the  cap- 
abilities of  the  three  indicated  submodels. 


SIEGEL-WOLF  INFORMATION  THEORETIC  MODEL 

A.  I.  Siegel,  J.  J.  Wolf  and  R.  Oilman  published  a report  in  1962 
which  laid  much  of  the  groundwork  for  the  subsequent  modeling  work  at 
Applied  Psychological  Services  as  well  as  providing  a starting  point 
for  other  man/machine  modeling  efforts. 24  While  subsequent  versions 
of  the  Siegel-Wolf  model  have  been  described  above,  the  1962  version 
included  significant  elements  which  do  not  appear  in  later  versions. 

Specifically,  the  1962  model  incorporated  a submodel  for  calculat- 
ing operator  action  times  from  extensions  of  information  theory.  Con- 
temporary experimental  psychologists  (notably  Crossman,  Fitts,  Hick  and 
Hyman)  had  used  the  "information  content"  of  stimulus  and  response  ele- 
ments as  a basis  for  predicting  stimulus-response  times  in  laboratory 
experiments.  Siegel  and  co-workers  generalized  from  this  work,  and 
produced  two  submodels  - for  "execution  time"  of  a control/display 
subtask,  the  other  for  commoaication  subtasks. 

All  the  models  reviewed  earlier  in  this  report  have  replaced  these 
submodels  for  task  execution  clme  with  "input  data"  from  unspecified 
sources.  When  directly  relevant  data  are  available  for  a task  being 
analyzed,  it  is  probably  preferable  to  use  the  data  rather  than  a 
model.  However,  the  application  of  modeling  to  advanced  cockpits  with 
unconventional  displays  and  controls  is  difficult  because  of  the  usual 
lack  of  directly  relevant  operator  time  data.  For  such  cases,  the  use 
of  the  information-theoretic  submodels  might  be  the  best  available 
option.  Limited  validation  results  reported  by  Siegel  in  the  refer- 
enced document,  plus  a small  amount  of  unpublished  validation  data 
collected  at  Autonetics,  lend  considerable  credibility  to  these  sub- 
models. 

The  Interested  reader  is  urged  to  consult  the  referenced  report 
for  a description  of  the  data  sources  used  and  the  formulations  de- 
rived from  chose  data. 


^Applied  Psychological  Services.  A Discontinuous  Analytic  Model  for 
Simulating  Apollo  Vehicle  Operator  Actions  and  Information  Exchange 
by  A.  I.  Siegel,  J.J.  Wolf  and  R.  Oilman.  Wayne,  PA.  September  19b2 
(publication  UNCLASSIFIED- ) 
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TABLE  6.  Elements  of  Cockpit  Evaluation  Process  (CAFES). 


Step  In  Process 

How  Handled 

External  to  Modal 

Mod. 

Internal 

1.  Define  Tasks: 

1.1  Mission  description 

Prepare  scenario; 

FAM-1 

Used  to  generate 

task  list,  objec- 

success  probe- 

tlves. 

blllty. 

Same  plus  rules 

FAM-2 

Uses  to  generate 

for  precedence. 

operational 

procedures. 

Same  plus  time 

WAM 

See  A. 

phasing . 

1.2  Cockpit  equipment 

Not  required  for 

description 

FAM  or  WAM. 

2.  Determine  task 

performance: 

2.1  Time  required  to 

Input  average 

FAM  & 

Scored  for  use 

perform  casks 

par  cask. 

WAM 

in  workload  and 
operational  pro- 
cedures . 

2.2  Accuracy /reliab- 

Input  error  data; 

FAM-1 

Combined  (with 

ility  of  opera- 

reliability  vs 

time)  to  give 

tor  action 

cask  loading 

mission  success 

data. 

probab ility . 

2.3  Task  sequences 

InpuC  precedence 

FAM-2 

Used  to  generate 

1 priorities 

rules;  input 

operational 

"priority  vs 
time  remaining" 
function. 

procedures . 

3.  Modify  performance 

1 

3.1  Effects  of  opera- 

Included  in  input 

tor  variables 

performance  data, 
if  at  all. 

3.2  Effects  of  environ- 

Same  as  above . 

aent 

4.  Determine  Operator 

Task  Load: 

[ 

i.l  Overall  workload 


Input  operator 
task  load  data 


WAM  S 
FAM-2 


Provides  work- 
load by  mission 


Step  in  Process 


External  to  Model 


WAM 

WAM 

FAM-2 

FAM-1 


How  Handled 


Internal 


Computes  "mis- 
slon  success" 
overall  and 
by  segment. 


Input  "activity 
by  channel"  for 
mission,  on  time 
base. 

Input  operator 
task. 


4.2  Workload  by  body 
element  or 
"channel" 


4.3  Effects  of  over- 
load 


Sequences  alter- 
ed if  overload 


occurs. 


Input  data  on 
operator  time/ 
accuracy  must 
reflect  arrange- 
ment. 


Performance 
5.1  Effects  of  cock- 
pit arrange- 
ment 
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TABLE  6.  (Continued.) 


Provides  percent 
workload  by 
channel  by  seg- 
ment. 


Overloads  flag- 
ged & sources 
indicated. 
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BASIS  FOR  COMPARISON  OF  MODELS 


The  purpose  of  the  descripclon  end  comparison  of  existing  crew/ 
cockpit  models  is  to  provide,  in  a single  report,  enough  information  to 
give  potential  users  or  modelers  a qualitative  picture  of  the  available 
models  and  their  areas  of  applicability.  It  is  expected  that  anyone 
seriously  considering  the  use  of  a model,  or  contemplating  the  construc- 
tion of  a new  or  improved  one,  will  consult  the  source  documentation. 

MODEL  CHARACTERISTICS 

Computer  models  of  systems  can  be  classified  and  compared  on  the 
basis  of  a number  of  characteristics,  some  broadly  fundamental  and 
others  more  specific  to  the  crew/cockpit  system.  The  characteristics 
considered  here  are  drawn  primarily  from  recent  treatments  by  Fishman25 
and  by  Siegel  and  Wolf. 26 


Deterministic  vs  Stochastic 

A deterministic  model  permits  the  calculation  of  a specific,  quan- 
titative solution  to  a set  of  input  conditions.  Stochastic  models 
provide  statistical  descriptions  of  a set  of  outputs  (such  as  the  mean 
and  variance) , but  provide  no  basis  for  computing  the  value  of  a parti- 
cular output  for  a set  of  Inputs. 

Few  crew  characteristics  lend  themselves  well  to  deterministic 
modeling.  This  limitation  leads  to  models  which  may  have  cascaded 
stochastic  elements.  An  exception  is  the  HOS  model;  although  not  com- 
pletely deterministic,  it  is  assumed  that  the  variability  in  certain 
operator  characteristics  is  insignificant  for  total  variability  in  cask 
performance.  Reach,  grasp,  and  scan  can  be  adequately  modeled  by 
deterministic  approaches,  whereas  cognitive  functions  are  more  likely 
to  require  stochastic  modeling. 

Slegel-Wolf-U/DOSM.  This  model  is  largely  stochastic.  Input  data 
include  distribution  of  times  and  likelihood  of  error,  by  cask,  sampled 
in  multiple  runs.  Output  includes  performance  statistics. 

McDonnell  Douglas-PSM.  This  model  is  also  mainly  stochastic. 

Input  data  includes  distribution  of  times  and  likelihood  of  error,  bv 
cask,  sampled  in  multiple  runs.  Output  includes  mean  and  standard 
deviation  of  task  times  by  cask. 


2S 

G.S.  Fishman.  Concepts  and  Methods  in  Discrete  Event  Digital  Simu- 
lation. New  York,  Wiley,  1973. 

A.E.  Siegel  and  J.J.  Volf.  Man-Machine  Simulation  Models.  New  York, 
Wiley,  1969.  
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Vought-WRM.  This  modal  is  largaly  dacarainiscic . Task  cimas  ara 
input  as  singla  valuas.  Tima  strass  is  computad,  onca  par  second,  as 
a singla  valua.  Discrata  task  tima  strass  is  calculated  on  a potential 
worst-case  basis  using  aarliest-time-of-initiation,  latest-time-of- 
completion  subroutine  to  eliminate  sequence  problems. 

MADC-HOS  - This  modal  is  also  largaly  deterministic . Excapt  for 
elemental  absorption  and  calculation  tlmas,  no  task  times  ara  inputs  to 
HOS.  Timas  are  determined  by  the  modal  from  the  microchange  times.  The 
modal  for  short-term  recall  ("hab")  is  explicitly  stochastic.  Outputs 
ara  simply  times,  accuracies,  sequences  to  perform  and  tlmas  spent  at 
each  activity  or  procedure. 

Boeing -CAFES  - The  Workload  Assessment  Model  (WAM)  of  the  CAFES 
series  is  a mixed  model.  Timas  for  tasks  are  put  in  as  nominal  values, 
sequenced  according  to  scenario.  Task  load  is  sampled  at  selected 
intervals;  output  as  mean  and  sigma  of  workload. 

General  vs  Specific 

Generality  in  a model  Is  achieved  when  a new  situation  can  be 
modeled  without  substantial  changes  to  the  model  structure.  Most  crew/ 
cockpit  models  are  rather  general  because  they  are  driven  by  a scenario 
and  task  list,  both  quite  flexible,  and  because  the  quantitative  rela- 
tionships are  usually  presented  as  tabular  data  from  field  or  simulator 
tests.  As  new  conditions  are  specified,  new  data  sources  are  used  as 
Inputs.  Generality  typically  Increases  further  as  the  model  becomes 
more  microscopic,  if  it  can  be  assumed  that  any  action  can  be  repre- 
sented as  a sequence  of  simple  reflex-arc  elements. 

Slegel-Wolf-U/DOSM.  This  model  is  quite  general  in  structure. 

New  conditions  are  accommodated  by  encerlng  new  data  on  operator, 
mission,  equipment  and  performance.  Algorithms  for  stress,  goal 
aspiration,  etc.  are  unchanged. 

McDonnell  Douglas-PSM.  Tills  model  is  quite  general,  but  new  input 
data  are  required  for  any  system  change.  Only  a few  items  (e.g.,  G- 
penaltles,  reach  distance  effects)  are  assumed  to  apply  to  new  cases. 

Vought-WRM.  This  model  is  also  quite  general,  but  requires  new 
input  data  for  any  system  change.  Only  a few  items  (e.g.,  reach  time 
vs  distance)  are  assumed  to  apply  unchanged  to  new  cases. 

NADC-HOS . Quite  general,  for  cases  which  can  be  presumed  to  be 
made  up  of  the  some  "units"  (e.g.,  look  at,  grasp),  with  the  same 
modifiers  (e.g.,  0-status,  "hab"  strength). 

3oelng-CAFES . FAM-1  and  WAM  are  quite  general,  because  these 
model  elements  are  basically  computer-based  aids  to  the  manual  "cask 
analysis"  and  related  processes. 
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Holistic  vs  Atomistic 

Man/machina  incaractlon  avancs  can  ba  daacrlbad  (hanca  modalad) 
aichar  aa  a series  of  micro-events  (a.g.,  "flxata,"  "reach,"  "graap") 
or  as  smooch,  lncagrated  aces  (a.g.,  "adjust  fraquancy") . 

Siegel-Wolf-U/DOSM.  Thia  modal  la  modaracaly  acomlaclc.  Tha  sub- 
cask  alamanes  which  maka  up  cha  modal  lncluda  such  aceions  aa  saeeing 
a rocary  conerol  or  chack-raadlng  a sac  of  lnscrumancs. 

McDonnall  Douglas-PSM.  This  modal  la  also  modaracaly  acomlaclc. 

All  caaka  (a.g.,  "adjust  chroccla")  ara  furchar  brokan  down  lnco  Infor- 
maclon-Procasalng-Acdon  aaquancas . 

Vought-WRM.  This  modal  la  mora  hollsdc  Chan  mosc.  Data  ara 
scorad  and  procasaad  at  cha  "task"  laval  (a.g.,  "obsarve  HUD  alrspaad," 
"mova  chroccla  to  k Mil"). 

NADC-HOS . Tha  basic  alemancs  are  acomlaclc  (a.g.,  "reach," 

"graap") . Combined~~unit*  which  ara  often  repeated  are  also  used,  how- 
ever. 

Boeing  - CAFES.  Thia  model,  like  the  WRM,  scores  and  proceasas 
data  ac  Cha  "cask"  level  (e.g.,  "scan  for  threats, " "change  heading"). 

Analytical  vs  Numerical 

An  analytical  model  is  one  which  can  be  expressed  in  algorithms  or 
closed  analytical  form,  permltcing  precise  calculation  of  outputs  from 
known  inpuc  quantities.  Numerical  models  represent  relationships  among 
quantities  as  look-up  Cables  or  ocher  non-closad  formats.  Most  aspects 
of  operator  performance  are  difficult  to  represent  by  simple  analytic 
forms,  and  ara  represented  by  tables.  In  the  following  comparisons, 
only  those  items  which  are  represented  by  algorithms  are  called  out. 
Other  relationships  are  handled  by  tables 

Slagel-Wolf-U/DOSM.  "Time  stress,"  "Goal  Aspiration,"  and  "Co- 
hesiveness" (for  a two-man  crew)  are  computed  by  algorithm. 

McDonnell  Douglas-PSM.  Performance  decrements  (from  t ime-penal c ies 
to  blackout)  are  compuced  as  functions  of  G-loads.  Also,  time  penalties 
for  reach  discance  are  computed. 

Vought-WRM.  Reach-time  and  look-time  are  computed,  aa  a function 
of  instrument  location.  Total  workload  is  computed  from  continuous  task 
and  discrete  cask  loads  by  an  algorithm. 

NADC-HOS . Reach,  grasp,  and  manipulation  times  are  computed  by 
an  "anatomy  mover"  module.  Operator  tnemorv  is  modeled  explicitly,  and 
used  to  determine  whether  a display  can  be  "remembered."  Criticality 
of  casks  is  computed  as  a function  of  mission  state  and  structured 
as  an  algorithm. 

3oelng-C.VFES . In  the  FAM-1  module,  operator  reliability  is  com- 
puted as  a function  of  task  load  and  ocher  variables. 
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Continuous  vs  Diacre ca-Event 

A continuous  modal  is  ona  in  which  system  conditions  are  bast 
represented  as  time-varying  quantities  which  can  have  any  value,  with- 
in specified  limits.  In  discrete-event  models,  the  system  condition  is 
best  described  as  one  of  a number  of  states. 

Crew/cockpit  models  generally  contain  some  elements  which  lend 
themselves  easily  to  continuous  modeling,  but  also  include  large  num- 
bers of  discrete-state  elements.  The  blending  of  these  elements  is  one 
source  of  variation  in  treatment  among  the  models  reviewed  here.  The 
handling  of  continuous  tasks  as  such  is  not,  however,  a topic  of  the 
present  study. 

In  all  the  models  described  here,  continuous  tasks  are  blended  in- 
to discrete  activities  by  viewing  them  as  a sequence  of  check-readings 
and  corrective  actions,  in  one  form  or  another.  The  most  explicit 
statement,  and  the  method  for  blending  the  two,  appears  to  be  found 
in  che  Vought  model. 

Parallel  vs  Serial  Operation 

The  operator  in  a complex  aircraft  system  performs  many  separate 
casks,  using  different  limbs  or  sense  organs.  Some  of  these  casks 
overlap  in  time.  The  representation  of  these  overlapping  tasks 
requires  a decision  as  to  whether  che  operator  can  be  represented  as 
performing  two  acts  simultaneously  or  not,  and  a mechod  of  determining 
cask  loading  in  Che  multi-task  cases. 

Siege 1-Wolf -U/POSM.  Tasks  for  an  individual  operator  are  modeled 
serially. 

McDonnell  Douglas-PSM.  This  model  permits  simultaneous  casks  if 
they  use  a different  sensor  or  effector.  Task  load  may  go  >100*;  then, 
based  on  task  priority,  automatically  task  reordering  begins. 


Voughc-WDM.  This  model  also  permits  simultaneous  "asks  up  to  an 
empirically  determined  workload  limit  which  is  based  on  che  discrete- 
continuous  cask  mix  ratio.  For  example,  the  short-term  limit  for  a cask 
situation  consisting  of  discrete  tasks  only  is  2301  at  the  nominal  long- 
term work  capability.  Lower  values  are  established  for  other  ratios. 
Beyond  these  level3  che  model  delays  least  critical  Casks  to  keep  with- 
in che  workload  limit. 

NADC-HOS . This  model  permits  simultaneous  actions  if  in  different 
"faculties.”  Tasks  can  be  delayed,  omitted,  or  reprioricized  as  a 
function  of  criticality. 

3eeing-CAFES . The  WAM  nodule  of  CAFES  permits  simultaneous  casks, 
flags  overloads,  and  can  use  automatic  task  shifting  if  wanted. 


J0 
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Prioritizing  and  Qmmlia 

In  aosc  cases  of  interest,  tasks  will  arise  more  rapidly  than 
they  can  be  performed  during  certain  critical  mission  phases.  The 
queueing  of  unperformed  tasks  can  be  strictly  first-come,  first-served, 
or  tasks  can  be  prioritized  and  can  even  interrupt  ongoing  tasks,  with 
the  concomitant  requirement  for  classification/decision  rules. 


Slcgel-Wolf-U/DOSM.  Each  subtask  is  given  an  "essentiality"  rat- 
ing, and  may  have  associated  with  it  certain  precedent  and  subsequent 
conditions.  Low-essentiality  tasks  can  be  omitted  if  time  stress  is 
too  high. 

McDonnell  Douglas-PSM.  "Criticality"  of  each  task  is  an  input 
(integer  1-4).  Each  task  input  also  Includes  a list  of  casks  which 
it  will  interrupt. 

Vought-WRM.  A level  of  "priority"  is  assigned  to  each  cask  (as 
an  integer  1-5);  "frequency"  of  each  cask  varies  with  mission  segment. 
Sequence  is  sec  using  Monte  Carlo  techniques,  based  on  "criticality" 
and  "frequency"  of  each  cask.  Error  nulling  casks  cake  priority  over 
discrete  casks. 

NADC-HOS . "Criticality"  of  each  concrol/display  varies  with 
mission  phase  and  time  since  last  checked.  Sequence  of  acts  depends 
upon  the  initial  state  of  the  device  as  well  as  operator  state  and 
mission  phase.  Criticality  can  be  changed  by  procedures  during  model- 
ing based  on  events  or  quantities  available  to  the  model. 

Boeing-CAFES . FAM-2 . Each  cask  carries  a "priority"  (3  Levels)  and 
an  "incerruptabilicy"  (3  levels)  rating.  A set  of  rules  (e.g.,  pre- 
requisite tasks,  mandatory  casks,  time  since  latest  check)  determines 
sequence  for  operational  procedures. 

Handling  of  Performance  Variability 

Variability  in  performance  of  a repeated  cask  is  characteristic 
of  humans.  The  modeling  of  this  variability  can  be  on  a mechanistic, 
random  basis,  or  an  attempt  can  be  made  to  model  variability,  wholly 
or  in  part,  in  terms  of  causes  (e.g.,  fatigue,  stress,  training, 
attention) . This  characteristic  is  a restatement  and  particulariza- 
tion of  Che  "Deterministic  vs  Stochastic"  descriptor  discussed  earlier 
in  this  section. 

Siegel-Wolf-U/DOSM.  This  model  makes  a considerable  effort  to 
accommodate  performance  variability  both  between  and  within  operators. 
Individual  operators  are  characterized  by  a "stress  threshold,"  an 
individual  "speed"  parameter,  and  an  initial  level  of  "goal  aspira- 
tion." Variability  within  an  operator  during  the  mission  is  handled 
in  part  by  algorithm  (e.g.,  stress  level,  goal  aspiration  level)  and 
in  part  by  random  draws  from  performance  distributions. 
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McDonnell  Douglas-PSM.  Variability  due  co  certain  environmental 
effects  (e.g.,  G-load,  long  reach)  are  handled  as  explicit  effects. 
Task  time  variability  due  to  other  causes  is  determined  by  Monte  Carlo 
methods,  from  stored  distributions.  Output  statistics  are  based  on 
multiple  (with  a limit  of  100)  replications  of  a model  run. 

Vought-WRM.  Variability  due  to  some  effects  (G-load,  visibility, 
reach  distance)  is  handled  explicitly.  Random  variability  is  modeled 
only  in  the  distribution  of  tasks  within  time  "windows."  Task  per- 
formance time  Itself  is  assumed  to  be  fixed. 

NADC-HOS . In  theory,  all  variability  is  to  be  computed  from  "0- 
state"  habit  strength,  environment,  initial  state  of  displays/controls, 
etc.  In  practice,  submodels  for  some  of  these  effects  have  not  been 
developed . 

Boeing-CAFES . Variability  from  run  to  run  is  not  represented, 
except  as  system  or  rule  structure  is  changed. 

Handling  of  Decisions 

A crew  member  who  is  monitoring  the  status  of  one  or  more  system 
indicators  must  have  available  in  some  form  a set  of  limits  or  other 
criteria  for  deciding  when  the  indicated  system  status  requires  action. 
Various  kinds  of  internal  models  and  decision  functions  have  been  post- 
ulated in  modeling  the  human  monitor. 


Siegel-Wolf-tl/DOSM.  "Decisions"  are  represented  explicitly  only 
in  the  "decision"  task  type,  which  is  a means  of  providing  branching 
in  the  analysis.  However,  "decisions"  about  urgency,  skipping  non- 
essential  tasks,  "trying  harder"  (goal  aspiration)  are  represented  by 
associated  algorithms. 

McDonnell  Douglas-PSM.  "Decisions"  relative  to  task  sequence  are 
described  under  "prioritizing,"  above.  The  information  processing 
element  of  each  task  addresses  which  decision  logic  will  be  used. 

Vought-WRM.  "Decisions"  relative  to  task  sequence  are  based  upon 
priority  rules  plus  Monte  Carlo  techniques.  Frequency  of  check  read- 
ings also  varies  as  critical  time  nears. 


NADC-HOS . "Decisions"  related  to  choice  of  task  (e.g.,  remember 
current  setting  vs  make  a check-reading),  as  well  as  task  sequence,  are 
based  upon  decision  rules  which  are  modeled  explicitly. 

Boeing-CAFES . FAM-2.  Decisions  relative  to  task  sequence  are 
based  upon  priority  (see  above)  plus  a "Remaining  Number  of  Opportun- 
ities" factor  which  "forces"  decisions  as  critical  time  nears. 


MODELING  SPECIFIC  COCKPIT-RELATED  EFFECTS 

The  Contract  Statement  of  Work  specifies  several  features  to  be 
evaluated  explicitly:  (1)  monitoring,  ^ 2 ) interactive  control  perform- 
ance with  integrated  displays,  and  (3)  color  coding.  These  three  spec- 
ific factors  are  discussed  below. 


•A2 


NWC  TP  6020 


i 


Ia  this  discussion,  the  term  "monitoring"  is  interpreted  as  syn- 
onymous with  "check-reading."  A display  is  checked  periodically  Co 
see  whether  it  is  in  a satisfactory  range,  or  requires  some  corrective 
action. 

The  far  more  complex  process  of  monitoring  the  state  of  the  system 
or  of  the  ongoing  mission  is  discussed  under  the  broader  heading  of 
"Integrated,  Interactive  Controls /Displays." 

Handling  of  Simple  Monitoring  Functions 

The  increasing  automation  in  avionics  equipment  has  given  rise  to 
an  increase  in  crew  casks  which  Involve  monitoring  an  automatic  process. 
This  development  and  some  of  its  implications  were  discussed  in  the 
introduction  to  this  report.  Here  the  five  models  studied  are  compared 
in  terms  of  their  handling  of  simple  monitoring  casks. 

Slegel-Wolf  Model.  Monitoring  or  check-reading  tasks  are  modeled 
explicitly  in  the  Siegel-Wolf  U/DOSM  model.  The  frequency  of  check- 
reading  actions  is  affected  by  the  input  "essentiality"  of  the  task, 
and  by  precedence  rules.  Check-reading  actions  which  depend  upon  an 
equipment  cycle  (e.g.,  monitoring  a PPI  radar  display)  are  assessed 
with  appropriate  time  delays. 

McDonnell  Douglas-PSM.  Although  all  casks  are  defined  as  I-P-A 
cycles  (information/ processing/action) , provision  is  made  for  skipping 
Che  "action"  portion  if  no  operator  action  is  required.  Other  chan 
this  adjustment,  check-reading  or  monitoring  casks  are  handled  like  any 
others,  in  terms  of  variability  of  performance  and  the  effects  of 
mission  conditions. 

Voughc-WRM.  Check  reading  or  monitoring  is  modeled  explicitly  as 
a part  of  the  continuous  control  portion  of  the  model.  Frequency  of 
monitoring  is  varied  as  a function  of  cask  criticality,  control  toler- 
ances, and  stability  of  the  controlled  element. 

Other  non-concinuous  monitoring  casks  (e.g.,  checking  communica- 
tion channel  selection)  are  handled  as  any  ocher  discrete  tasks. 

NADC-HOS . The  HOS  model  literature  makes  explicit  reference  to 
check-reading  casks,  and  provides  a discussion  of  many  of  che  relevanc 
variables.  However,  as  of  May,  1975,  the  status  of  the  check-reading 
module  is  stated  thus  (in  "Development  of  a Quantitative  Display  Read- 
ing for  HOS,"  Analytics  TR  1117-F,  by  M.I.  Streib,  p.4-2): 

"We  intend  eventually  to  develop  a separate  model  for 
performance  of  check-reading  casks." 

Boeing-CAFES . Both  FAM  and  WAM  modules  require  detailed  descrip- 
tions of  all  casks  including  racings  of  interruptabili cy , criticality, 
concentration,  etc.,  as  well  as  performance  data.  Monitoring  or  check- 
reading  casks  would  have  co  be  handled  in  the  same  way  as  any  ocher 
casks,  with  che  attendant  requirement  for  estimation  of  all  cask  des- 
criptors for  such  casks. 
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Handling  of  Integrated.  Interactive  Controls/Displays 

The  tarn  "interactive"  has  not  bean  tightly  defined  in  the  litera- 
ture, but  has  come  to  be  used  in  several  different  ways.  Three  possi- 
ble levels  of  "Interactiveness"  follow. 

Feedback  from  Operator  Actions.  Probably  the  simplest  level  of 
"Interaction"  without  integration  is  one  in  which  the  control/display 
system  provides  feedback  to  the  operator  when  he  actuates  a control. 
However,  this  level  of  interaction  is  practically  universal  in  military 
aircraft,  since  it  is  a requirement  of  Mil  Std  1472. 

Change  of  System  Configuration.  This  level  of  "interaction"  is 
achieved  when  a control  action  results  in  a reconfiguration  of  some 
part  of  the  system.  For  example,  selection  of  a weapon  delivery  mode 
may  result  in  a reconfiguration  of  the  display  on  a HUD.  The  system 
has  performed  internal  reorganization  in  anticipation  of  particular 
operating  requirements.  The  result  is  a certain  amount  of  integration 
as  well,  since  multiple  functions  are  tied  together  in  a single  piece 
of  equipment. 

Tutorial  Display.  The  most  sophisticated  type  of  integrated 
interactive  system  is  one  which  guides  the  operator  through  a series 
of  steps  initiated  by  the  selection  of  an  operating  mode.  One  example 
is  the  reconfigurable  keyboard,  in  which  the  legends  change  depending 
upon  the  preceding  entry,  presenting  only  those  choices  which  are 
appropriate  at  the  moment.  Another  example  is  a schematic  or  diagram- 
matic display,  in  which  system  configuration  is  presented  in  such  a 
way  as  to  highlight  "next  step"  options  after  each  control  action. 

Implications  for  Modeling.  The  characteristics  which  set  "inte- 
grated interactive"  display/control  elements  apart  from  others  are 
somewhat  elusive.  However,  among  the  properties  which  might  serve  to 
distinguish  them  are: 

1.  Control/display  functions  cannot  be  related  uniquely  to 
locations  in  the  cockpit,  or  conversely,  multiple  func- 
tions will  have  to  be  associated  with  a given  location. 

2.  "Display  reading"  as  a task  or  subtask  may  have  to  be 
modeled  differently  when  the  display  is  reconfigurable. 

Content  becomes  all-important. 

3.  Multi-path  task  sequences  will  have  to  be  provided  in  the 
modal.  This  is  not  a new  requirement  since  even  the  simplest, 
hard-wired  avionics  system  will  permit  alternative  scenarios, 
but  the  branching  will  be  more  pervasive. 
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Examples  of  Integrated,  Interactive  System  Tasks.  In  all  the  cock- 
plt/operator  models  described  in  this  report,  a detailed  list  of  tasks 
and  task  options  Is  required  to  "drive"  the  model.  The  difference  be- 
tween dedicated  display/control  equipment  and  integrated,  interactive 
systems  will  be  reflected  in  the  nature  of  the  tasks. 

A brief  task  sequence  is  presented  in  Table  7 for  an  A7-E  aircraft 
with  conventional  avionics,  and  with  an  integrated,  interactive  con- 
trol/display sat.  It  will  be  noted  that,  for  this  sequence,  the  nature 
of  the  tasks  changes  substantially,  indicating  that  new  sources  of  time 
and  accuracy  data  will  be  required  when  modeling  the  newer  systems. 
However,  it  doesn't  appear  that  any  difference  in  model  structure  would 
be  required  for  this  type  of  integration. 

A higher  level  of  system  autonomy  is  planned  in  certain  future 
avionics  systems,  and  would  seem  to  require  a different  level  of  des- 
cription and  analysis.  An  example  might  be  an  "automated"  ECM/ECCM 
system.  Here  the  equipment  would  be  expected  to  sense  the  electronic 
environment  and  to  make  appropriate  responses,  based  upon  a set  of 
decision  criteria.  The  crew  might  normally  receive  only  status  infor- 
mation, such  as  threats  detected,  direction  of  each,  countermeasure 
being  used,  and  remaining  capacity  (such  as  chaff,  number  of  jammers, 
decoys) . 

Here  the  crew  member  is  expected  to  monitor  the  tactical  situa- 
tion, and  to  Intervene  only  when  he  feels  intervention  is  required. 

To  describe  this  relationship  as  a series  of  tasks,  with  associated 
performance  times  and  error  rates,  does  not  appear  on  the  surface  co 
be  feasible. 

Adaptability  of  Present  Models  - The  models  reviewed  here  vary 
somewhat  in  the  extent  to  which  they  are  adaptable  to  sophisticated 
interactive  designs.  The  quality  and  depth  of  the  task-analytic  work 
required  for  model  inputs  will  also  affect  the  handling  of  these  soph- 
isticated systems. 

1.  Slegel-Wolf  Model.  Interaction  at  the  simple  level  (i.e., 
system  responds  to  operacor  input;  operator  perceives  resulting  system 
change;  operator  selects  next  action  in  part  as  a function  of  perceiv- 
ed system  response)  is  handled  in  early  versions  of  the  model  by  a 
"decision  task"  and  a Monte  Carlo  choice  from  the  options  available 
at  the  decision.  A later  version  (1971)  adds  the  option  of  stopping 
the  simulation  at  a "decision"  poinc,  lecting  the  user  make  the  deci- 
sion, then  starting  again.  This  version  in  effect  "models"  the  deci- 
sion process  by  using  a surrogate  crew  member.  Presumably  a "tutorial" 
interactive  system  would  be  simulated  in  this  way.  However,  as  che 
system  becomes  more  autonomous,  che  "surrogate  crew  member"  might 
become  essential  throughout  che  simulation. 

The  effects  of  "incegraced  displays"  (i.e.,  multi-purpose,  recon- 
figurable)  are  not  modeled  explicitly,  but  would  have  co  be  reflected 
in  che  input  data. 
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"Interaction”  of  crew  members  with  mission  conditions  is  repre- 
sented dynamically  in  this  model  by  the  varying  values  of  "time  stress" 
and  "goal  aspiration." 

2.  McDonnell  Douglas  PSM.  Interaction  between  system/mission 
status  and  performance  is  provided  in  each  task  by  (1)  a variable  time 
delay,  to  allow  waiting  for  changes  in  status,  and  (2)  by  operator 
"probability"  values,  which  permit  recycling  or  other  alternative  paths, 
and  which  reflect  certain  limited  mission  conditions. 

Interaction  with  an  Integrated,  tutorial  control/display  system 
is  not  explicitly  considered,  although  if  the  task  analysis  and  OSD 
work,  done  before  the  model  is  run,  were  done  thoroughly  and  in  detail, 
the  effects  upon  performance  would  presumably  be  accurately  represent- 
ed, at  least  for  simpler  systems. 

Interaction  with  the  characteristics  of  the  aircraft  and  control 
system  can  be  handled  in  another  way  in  PSM.  The  model  can  be  tied 
to  a physical  aerodynamic  simulation,  providing  more  "real"  environ- 
mental effects,  since  PSM  can  operate  real-time. 

3.  Vought  WRM.  The  WRM  is  sensitive  to  certain  types  of  mission 
interaction  explicitly.  The  effect  of  "time-to-go"  upon  frequency  of 
check-reading  is  modeled,  on  a somewhat  intuitive  basis,  giving  an 
"urgency"  effect.  The  effect  of  mission  requirements  and  aerodynamics 
upon  control  tasks  (e.g.,  frequent  corrections  to  altitude  just  before 
a carrier  landing)  is  also  explicitly  modeled. 

The  effects  of  integrated,  interactive  display/control  equipment 
is  not  explicitly  present,  but  could  be  incorporated  by  means  of  a 
sufficiently  detailed,  valid  task  analysis,  so  far  as  that  technique 
is  valid. 

4.  NADC  HOS.  The  HOS  model  (with  the  related  HOPROC,  HAL  and 
HODAC)  is  designed  to  be  a flexible,  interactive  model,  hopefully  over- 
coming some  of  the  limitations  of  ths  other  operator  models.  Inter- 
action between  performance  and  mission  variables  (through  time-varying 
priorities  and  0-states)  and  flexibility  of  procedures  (through  op- 
tions such  as  recalling  a reading  vs  re-reading  it)  are  provided  for 

in  the  structure. of  HOS  (though  not  all  elements  have  been  construct- 
ed in  detail.) 
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5.  Boeing  CAFES.  As  indiesced  earlier  in  this  report,  CAFES  is 
an  assembly  of  models,  and  is  designed  to  incorporate  a HOS-like  opera- 
tor nodal  in  the  future.  The  existing  modules  all  rely  upon  detailed 
task  lists  and  sequence  rules  as  Inputs,  providing  opportunities  for 
modeling  interactive  systems.  Such  interactions  are  not,  however, 
explicitly  a part  of  the  CAFES  model. 


Handling  of  Color-Coding 


The  contract  statement  of  work  calls  for  specific  attention  to  the 
handling  of  color  coding  in  the  models.  It  appears  that  none  of  the 
models  has  explicit  provision  for  color  effects.  However,  if  perform- 
ance data  (e.g.,  reading  time  and  error  rates)  were  available  for  dif- 
ferent types  of  color  coding,  model  runs  could  be  made  showing  the 
effects  of  the  color  coding  changes. 


Thus,  none  of  the  models  has  a coding  color  effect  built  in,  but 
all  of  them  are  capable  of  showing  the  effects  of  color,  given  ade- 
quate input  data. 
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REPRESENTATION  OF  EQUIPMENT,  OPERATOR,  AND  MISSION  DATA  IN  THREE  MODELS 

Three  of  the  five  models  described  and  compared  above  are  actively 
being  used  to  represent  attack  aircraft  and  crews  In  complete  attack 
missions.*  They  are  "complete,"  In  a utilitarian  sense,  and  they  repre- 
sent three  available,  comparable,  alternative  models  for  the  same 
classes  of  mission  simulations. 

In  this  section,  these  three  comparable  models  ( Siegel -Wolf , 
McDonnell  Douglas  and  Vought)  are  compared  on  their  handling  of  a 
number  of  the  rather  mundane  variables  which  describe  "real"  cockpits, 
people,  and  missions. 

Table  8 is  related  to  equipment  characteristics.  It  will  be  noted 
that  (1)  most  characteristics  are  not  "modeled,"  but  must  be  represent- 
ed in  input  data,  if  at  all,  and  (2)  the  few  characteristics  which  are 
represented  explicitly  vary  from  model  to  model. 

Table  9 presents  a similar  comparison  related  to  operator  charac- 
teristics. The  situation  here  is  more  complex  because  some  operator 
characteristics  (e.g.,  fatigue,  learning)  would  be  expected  to  change 
during  the  course  of  the  mission.  These  factors  cannot  be  represented 
by  input  data,  but  must  have  time-related  algorithms  associated  with 
them.  Other  operator  characteristics  (e.g.,  reach  distance,  quickness) 
would  be  fixed  during  a mission,  and  could  be  reflected  in  input  data, 
in  the  same  way  as  equipment  characteristics. 

Table  10  is  a comparison  of  the  effects  of  mission  characteristics 
upon  the  models.  Here  again,  some  characteristics  (e.g.,  weather, 
defense  environment)  would  be  time-sensitive,  but  would  presumably  be 
predictably  tied  to  the  mission  phases  and  so  could  be  handled  as  input 
data  effects. 


*The  MADC  model,  HOS,  cakes  a different,  more  detailed  approach  to  simu- 
lation chan  the  other  models.  It  is  not  efficient  to  use  HOS  to  simulate 
an  entire  mission,  but  it  is  complete  enough  to  be  considered  utilitarian 
HOS  has  been  used  to  simulate  che  complete  hardware/ sof tware  systems  for 
the  LAMPS  helicopter  operator  (ATO)  for  specific  mission  segments,  ar.d  is 
being  used  for  P-3C  applications. 
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TABLE  8.  Comparative  Handling  of  Equipment  Characteristics. 


Equipment  Descriptor 

Siegel- 

Wolf 

McDonnell 

Douglas 

"PSM" 

Vought 

"WRM" 

DISPLAYS 

Distance  \ Location 

I* 

M(2) 

E 

Direction  f 

I 

E ’ 

I 

Size 

I 

1 (3) 

I 

Format 

I 

I 

I 

Intensity 

I 

I (3) 

I 

Color 

I 

I 

I 

Dynamic  properties 

E**&I(1) 

I 

I (5) 

Content 

I 

I 

I (5) 

CONTROLS 

Distance  \ 

I 

*1(2) 

E (6) 

Direction  j 

I 

E | 

I 

Size 

I 

I 

I 

Type  of  control 

I 

I 

I 

Force  required 

I 

I 

I 

Type  of  motion  req'd 

I 

I 

I 

Accuracy  req'd 

I 

I 

I 

LIFE  SUPPORT 

Seat  type/ angle 

I 

I (4) 

I 

Protective  clothing 

I 

I 

I 

GENERAL 

Equip. -related  time 

E (1) 

I 

I (7) 

Equip,  failure 

I 

I 

No  (8) 

* "I"  implies  that  this  factor,  if  represented  at  all,  must  be  implicit 
in  the  input  performance  data. 

**  "e"  implies  that  this  factor  is  handled  explicitly  in  the  model, 
numbered  notes  indicate  how  it  is  done. 


NOTES  RELATED  TO  TABLE  3 (EQUIPMENT) 

Slegel-Wolf  Model 

(1)  Time  required  for  equipment  to  function  or  to  cycle  is  an  input; 
cycle  times  are  Chen  used  explicitly  as  delays  where  appropriate. 

McDonnell  Douglas  "PSM" 

(2)  Location  is  an  input  for  each  item  by  "zone";  performance  time 
is  penalized  as  a function  of  equipment  location. 

(3)  The  model  will  output  a minimum  size  and  luminance  for  dis- 
played data,  as  a function  of  G-load,  etc. 
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(4)  Seat  angle  Is  an  input,  but  can  be  changed  (15s  or  65°)  and 
will  affect  performance. 

Vought  Model 

f(5)  The  Vought  WRM  explicitly  models  these  factors  in  the 
"continuous"  task  model,  but  not  the  "discrete"  one. 

(6)  Reach  times  are  computed  by  an  algorithm  which  reflects  con- 
trol locations.  Penalties  may  also  be  assessed  against  "action"  time, 
for  long  reaches. 

(7)  Equipment  time  delays  could  be  included  in  empirically  deter- 

1 mined  cask  times. 

(8)  For  discrete  tasks,  probability  of  correct  performance  (which 
could  include  equipment  failures)  is  not  used. 


TABLE  9.  Comparative  Handling  of  Operator  Characteristics. 


Operator  Descriptor 

Siegel- 

Wolf 

m 

Vought 

"WRM" 

PHYSICAL 

Size 

I* 

— 

i 

I 

Somatotype 

I 

i 

I 

Posture 

I 

E(10) 

I4E( 13) 

Physical  Stress 

I 

E(ll) 

I&E(19) 

Visual  functioning 

No 

I4E(12) 

I&E(19) 

BEHAVIORAL 

Proficiency 

E**&I(1) 

I 

I 

Attention 

1(2) 

If  13) 

No 

Fatigue 

No(3) 

1(13) 

No 

Psychol.  Stress 

E(4) 

1(13) 

No 

Errors/ Ac curacy 

E&I (5) 

E&I(14) 

No (20) 

Morale /Aspiration 

E(6) 

I or  No  (13) 

No 

Time  per  action 

E&I(7) 

1(15) 

I/E(21) 

SOCIAL 

Crew  Makeup 

I 

1(16) 

I 

Cohesiveness 

E(8) 

I/E(17) 

No 

Comnunl cation 

E ( 9) 

1(16) 

I 

* "I"  implies  that  this  factor,  if  represented  at  all,  must  be 


implicit  in  the  input  performance  data. 

**  "E"  implies  that  this  factor  is  handled  explicitly  in  the  model. 
Numbered  notes  indicate  how  it  is  done. 
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NOTES  RELATED  TO  TABLE  9 (OPERATOR) 

Slagel-Wolf  Modal 

(1)  Individual  "speed"  factor  is  an  input;  parfonnanca  is  modifiad 
by  axplicit  algorithms  ralatad  to  stress,  aspiration  level,  goal 
proximity. 

(2)  Variations  in  "attention"  are  implied  as  an  effect  of  stresa, 
goal  proximity. 

(3)  Performance  changes  due  to  fatigue  during  mission  are  not 
Included . 

(4)  Stress  is  a central  concept  in  U/DOSM  model.  Computed  aa  f 
(time  remaining,  stress  threshold,  task  type). 

(5)  Average  error  rate  an  input;  pass/ fail  on  a specific  run  is 
a random  draw. 

(6)  "Aspiration  level"  is  computed  continually  as  function  of  past 
performance,  stress,  etc. 

(7)  Mean  and  standard  deviation  of  expected  task  times  are  Input; 
random  draws  from  a truncated  distribution  generate  the  specific  task 
time  on  a given  run. 

(8)  "Cohesiveness"  of  a 2-man  crew  is  computed  continually  as  a 
function  of  stress  levels  of  each  member. 

(9)  Performance  on  "communication"  subcasks  handled  like  other 
tasks;  an  algorithm  for  time  as  a function  of  message  sice  is  used. 

McDonnell  Douglas  "PSM" 

(10)  Seat  back  angle  (15°  to  65#)  modifies  performance;  previous 
eye  or  hand  position  modifies  task  times. 

(11)  G-load  is  computed  continually;  an  algorithm  modifies  per- 
formance time  with  G.  Also,  see  note  (13). 

(12)  An  input  pattern  plus  current  estimates  of  head  orientation 
define  the  geometry  of  the  visual  field,  as  a function  of  posture  and 
point  of  regard;  used  for  search  time  estimation.  Also,  dark  adapta- 
tion as  a function  of  time  is  modeled  explicitly. 

(13)  Several  "environmental  stress"  factors  exist  as  inputs,  and 
car(  be  varied  from  cask  to  task  through  the  mission.  They  might  be 
uaed  for  factors  not  now  considered. 

(14)  Basic  performance  accuracy  is  an  input  by  cask;  a "learning 
curve"  can  modify  performance  during  the  mission. 

(.13)  Basic  performance  cime  is  input  as  a simplified  distribution, 
by  displav/concrol:  random  draws  are  made  to  obtain  specific  times. 


It 
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(16)  "PSM"  is  a multi-man  model;  crew  make-up  and  intra-crew 
communication  can  be  handled,  via  input  data. 


(17)  "Cohesiveness"  is  an  identified  input;  modifiable  during 
the  mission. 


Vought  "WRM" 


(18)  General  posture  is  not  considered;  hand  or  foot  position  at 
the  end  of  a task  affects  the  next  reach-time  explicitly. 


(19)  Penalties  are  applied  for  a hlgh-G  condition;  other  physical 
stresses  are  considered  if  time  decrements  can  be  determined. 


(20)  Accuracy  of  performance  on  continuous  tasks  is  modeled,  but 
is  not  part  of  this  evaluation. 


(21)  Median  time  per  task  is  an  Input;  modifications  for  reach 
distance  are  explicit  (see  note  18) . 


TABLE  10.  Comparative  Handling  of  Mission  Characteristics. 


Mission  Descriptor 

Siegel- 

Wolf 

McDonnell 

Douglas 

"PSM" 

Vought 

"WRM" 

IDENTIFICATION 

Mission  type 

I* ** 

I 

I 

Difficulty 

1(1) 

E(4) 

I 

Criticality 

1(1) 

E(4) 

I&E(9) 

TIME  CONSTRAINTS 

Total  time  available 

I 

I 

I 

Event  sequence 

I&E(2) 

I4E(9) 

Event  time  limits 

I&E(2) 

■ 

I4E(10) 

ENVIRONMENT 

\ 

Terrain 

Weather 

Defenses 

ECM 

^ 1(3) 

j K6) 

}' 

Flight  regime 

1 

E(7) 

' E( 11) 

Maneuver s/G-loads 

/ 

E ( 8 ) 

E(12) 

* "I"  implies  that  this  factor,  if  represented  at  all,  must  be 

implicit  in  the  input  performance  data. 

**  "E"  implies  that  this  factor  is  handled  explicitly  in  the  model. 
Numbered  notes  Indicate  how  it  is  done. 
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NOTES  RELATED  TO  TABLE  10  (MISSION) 

Siegel-Wolf  Model 

(1)  These  characteristics  would  be  reflected  only  in  the  "avail- 
able  time"  and  "essentiality"  inputs. 

(2)  Subtask  sequence  and  time  are  input,  but  also  subject  to 
modification  during  a run  as  a function  of  "random  draw"  variation  in 
performance  time  and  consequent  time  stress;  non-essential  tasks  can 
drop  out. 

(3)  None  of  these  factors  is  input  per  se;  they  could  affect  per- 
formance time  and  accuracy  inputs,  and  task  sequence  and  essentiality. 

McDonnell  Douglas  "PSM" 

(4)  "Criticality"  and  "Difficulty"  are  input  for  each  cask;  allow- 
able values  are  1 to  4 (criticality)  and  1 to  3 (difficulty).  These 
values  will  affect  che  way  the  simulation  runs. 

(5)  Event  times  and  sequence  - see  note  (2)  . 

(6)  Values  of  "stress,"  "altitude,"  "temperature,"  factors 
can  be  entered  at  each  cask;  they  could  reflect  these  environmental 
characteristics . 

(7)  "Altitude"  and  "temperature"  are  input;  they  are  used  in  com- 
puting maneuver  envelopes,  etc. 

(8)  Predicted  G-loads  are  used,  with  algorithms  in  the  model,  to 
modify  sensory  and  motor  cask  performance. 

Vought  "WRM" 

(9)  "Criticality"  is  an  input  for  each  task;  sequencing  of  tasks 
is  based  partly  on  this  value. 

(10)  Event  time  and  sequence  - see  note  (2). 

(11)  Flight  regime  is  modeled  explicitly  in  the  "continuous"  por- 
tion of  che  model;  workload  for  all  casks  is  affected. 

(12)  Above  a threshold  level,  performance  decrements  are  assessed. 
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CONCLUSIONS  AND  RECOMMENDATIONS 


SUMMARY  OF  MODELING  WORK 

The  review  presented  in  this  report  leads  to  a number  of  conclu- 
sions and  generalizations  concerning  crew/ cockpit  modeling  efforts. 

They  are  stated  briefly  here.  It  must  be  reiterated  that  the  sheer 
volume  and  complexity  of  the  model  documentation,  and  the  fluidity  of 
most  of  the  models,  make  any  generalizations  somewhat  risky.  It  cannot 
be  too  strongly  stated  that  anyone  seriously  contemplating  substantial 
work  in  this  kind  of  modeling  should  study  the  original  documentation 
before  making  choices  and  decisions. 


Two  Modeling  Approaches 

The  apparent  motivation  behind  the  development  of  the  McDonnell  and 
Vought  models  (and  to  a lesser  extent  the  Siegel-Wolf  models)  was  to 
get  a good,  dynamic  representation  of  the  man-machine  system  "on  line" 
at  as  early  a date  as  practical.  This  has  been  accomplished  with  con- 
siderable success. 

The  apparent  motivation  behind  Wheery’s  effort  (the  HOS  complex) 
and  perhaps  the  Boeing  CAFES  and  Air  Force  SAINT  work  as  well,  has  been 
to  develop  a framework  with  enough  depth,  breadth  and  flexibility  to 
handle  a wide  variety  of  present  and  future  modeling  needs.  This  kind 
of  effort  is,  by  its  nature,  somewhat  open-ended;  consequently,  it  is 
almost  impertinent  to  ever  expect  such  a model  (or  collection  of 
models)  to  be  "completed."  The  ground  covered  during  the  development 
of  the  models,  however,  gives  rise  to  a variety  of  important  questions 
concerning  the  nature  of  the  man/machine  system.  These  questions  must 
be  faced  if  a deeper  understanding  of  the  system  is  to  be  achieved. 

The  Importance  of  Analysis 

None  of  the  models  takes  the  place  of  careful,  detailed  scenario 
development  and  task  analysis.  Indeed,  the  required  model  inputs 
force  the  analyst  to  work  through  mission  and  system  options  in  con- 
siderable depth. 
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jortance  of  Performance  Data 


Vith  the  exception  of  the  Siegel-Wolf  Information  theoretic  work 
for  Apollo,  all  the  models  depend  exclusively  on  performance  data  for 
the  basic  task  times  and  accuracies.  In  addition,  the  form  of  the 
"corrections"  and  "penalties"  which  are  Imposed  by  the  models  to 
account  for  G-load,  stress,  cockpit  layout,  etc.,  must  be  validated 
against  performance  data  if  they  are  to  be  used  with  confidence.  In 
some  cases,  validation  presents  a truly  formidable  cask,  since  it 
involves  factors  (such  as  fatigue,  level  of  aspiration,  time  stress) 
which  are  notoriously  difficult  to  measure  singly,  let  alone  In  com- 
bination. 


The  Elusiveness  of  Interactive  Control 

The  paradigm  lying  behind  the  modeling  of  cockpit  operations  Is 
generally  task-oriented.  The  operator  senses  a condition,  processes 
the  Information,  and  acts  on  the  system.  In  sequence.  Some  parallel 
tasks  are  permitted,  and  branching  is  accommodated,  but  the  basic 
stimulus  - response  structure  Is  dominant.  Whether  this  structure  is 
sufficiently  flexible  to  represent  adequately  the  kind  of  "supervisory 
control"  which  is  envisioned  for  future  attack  aircraft  remains  an  open 
question. 


RECOMMENDATIONS 

The  practicability  of  computer  modeling  of  the  cockpit/crew  system 
in  attack  aircraft  has  been  adequately  demonstrated.  Gradual  develop- 
ment of  the  existing  models  continues,  but  the  basic  structures  and 
related  software  realizations  appear  to  be  well  matured. 

The  major  areas  requiring  new  or  increased  work  appear  to  this 
reviewer  to  be  two: 

1.  Improvements  are  needed  in  the  formulation  and  validation  of 
the  modifiers,  penalties,  and  corrections  which  have  been 
involved  to  bridge  the  gap  between  laboratory  or  simulator 
data  on  human  performance  and  the  "real  world"  of  combat 
aviation.  All  of  the  models  incorporate  some  of  these  effects, 
often  on  the  basis  of  intuitive  sub-models,  but  the  relative 
importance  of  different  effects,  and  their  interactions  in 
real-world  settings,  remains  elusive. 

2.  Careful  study  of  the  operator  role  in  proposed  advanced, 
integrated,  interactive,  highly  automated  systems  needs 

to  be  undertaken.  The  NATO  Conference  report  cited  in  the 
Introduction  is  an  indication  that  this  effort  is  indeed 
under  way. 


